text2cinemagraph
Text2Cinemagraph: Text-Guided Synthesis of Eulerian Cinemagraphs [SIGGRAPH ASIA 2023]
파일 탐색기
최종 버전 다운로드 (.zip)- 0.png
- cap1.png
- cap2.png
- cap3.png
- caption1.png
- caption2.png
- control1.gif
- control2.gif
- demo.gif
- mask_odise.png
- mask_self_attn_erosion.png
- method.gif
- method_recording.gif
- sample_0.png
- synthesized_flow.jpg
- video.gif
- video2.gif
- video3.gif
- v1-inference.yaml
- inference.yaml
- inference_directional.yaml
- __init__.cpython-38.pyc
- __init__.cpython-39.pyc
- base_data_loader.cpython-38.pyc
- base_data_loader.cpython-39.pyc
- base_dataset.cpython-38.pyc
- base_dataset.cpython-39.pyc
- custom_dataset.cpython-38.pyc
- custom_dataset.cpython-39.pyc
- custom_dataset_data_loader.cpython-38.pyc
- custom_dataset_data_loader.cpython-39.pyc
- data_loader.cpython-38.pyc
- data_loader.cpython-39.pyc
- image_folder.cpython-38.pyc
- image_folder.cpython-39.pyc
- __init__.py
- base_data_loader.py
- base_dataset.py
- custom_dataset.py
- custom_dataset_data_loader.py
- data_loader.py
- image_folder.py
- prompts.txt
- prompts_twin.txt
- file2captions-eularian-train-blip2-20-15.txt
- file2captions-eularian-validation-blip2-20-15.txt
- generate_flow_hint.py
- nouns.json
- compute_fvd.py
- our_fvd.py
- util.py
- pnp_utils.cpython-38.pyc
- pnp_utils.cpython-39.pyc
- run_features_extraction.cpython-38.pyc
- run_features_extraction.cpython-39.pyc
- run_pnp.cpython-38.pyc
- run_pnp.cpython-39.pyc
- util.cpython-38.pyc
- util.cpython-39.pyc
- autoencoder.cpython-38.pyc
- autoencoder.cpython-39.pyc
- __init__.cpython-38.pyc
- __init__.cpython-39.pyc
- ddim.cpython-38.pyc
- ddim.cpython-39.pyc
- ddpm.cpython-38.pyc
- ddpm.cpython-39.pyc
- __init__.py
- classifier.py
- ddim.py
- ddpm.py
- plms.py
- autoencoder.py
- attention.cpython-38.pyc
- attention.cpython-39.pyc
- ema.cpython-38.pyc
- ema.cpython-39.pyc
- x_transformer.cpython-38.pyc
- x_transformer.cpython-39.pyc
- __init__.cpython-38.pyc
- __init__.cpython-39.pyc
- model.cpython-38.pyc
- model.cpython-39.pyc
- openaimodel.cpython-38.pyc
- openaimodel.cpython-39.pyc
- util.cpython-38.pyc
- util.cpython-39.pyc
- __init__.py
- model.py
- openaimodel.py
- util.py
- __init__.cpython-38.pyc
- __init__.cpython-39.pyc
- distributions.cpython-38.pyc
- distributions.cpython-39.pyc
- __init__.py
- distributions.py
- __init__.cpython-38.pyc
- __init__.cpython-39.pyc
- modules.cpython-38.pyc
- modules.cpython-39.pyc
- __init__.py
- modules.py
- __init__.py
- bsrgan.py
- bsrgan_light.py
- utils_image.py
- __init__.py
- contperceptual.py
- vqperceptual.py
- attention.py
- ema.py
- x_transformer.py
- lr_scheduler.py
- util.py
- autoencoder_kl_16x16x16.yaml
- autoencoder_kl_32x32x4.yaml
- autoencoder_kl_64x64x3.yaml
- autoencoder_kl_8x8x64.yaml
- celebahq-ldm-vq-4.yaml
- cin-ldm-vq-f8.yaml
- cin256-v2.yaml
- ffhq-ldm-vq-4.yaml
- lsun_bedrooms-ldm-vq-4.yaml
- lsun_churches-ldm-kl-8.yaml
- txt2img-1p4B-eval.yaml
- feature-extraction-generated.yaml
- feature-extraction-real.yaml
- feature-pca-vis.yaml
- pnp-generated.yaml
- pnp-real.yaml
- setup.yaml
- v1-inference.yaml
- .DS_Store
- dependency_links.txt
- PKG-INFO
- SOURCES.txt
- top_level.txt
- pnp_utils.py
- run_features_extraction.py
- run_pnp.py
- setup.py
- __init__.cpython-38.pyc
- __init__.cpython-39.pyc
- batchnorm.cpython-38.pyc
- batchnorm.cpython-39.pyc
- comm.cpython-38.pyc
- comm.cpython-39.pyc
- replicate.cpython-38.pyc
- replicate.cpython-39.pyc
- __init__.py
- batchnorm.py
- batchnorm_reimpl.py
- comm.py
- replicate.py
- unittest.py
- __init__.py
- attention.py
- base_model.py
- configs.py
- models.py
- networks.py
- normalization.py
- partialconv2d.py
- pix2pixHD_model.py
- softsplat.py
- coco_panoptic_semseg.py
- pano_open_d2_eval.py
- mask_generator_with_caption.py
- mask_generator_with_label.py
- odise_with_caption.py
- odise_with_label.py
- optim.py
- schedule.py
- train.py
- odise_caption_coco_50e.py
- odise_label_coco_50e.py
- ade20k_instance_catid_mapping.txt
- ade20k_instance_imgCatIds.json
- prepare_ade20k_full_sem_seg.py
- prepare_ade20k_ins_seg.py
- prepare_ade20k_pan_seg.py
- prepare_ade20k_sem_seg.py
- prepare_coco_caption.py
- prepare_coco_semantic_annos_from_panoptic_annos.py
- prepare_lvis_openseg_labels.py
- prepare_pascal_ctx_full_sem_seg.py
- prepare_pascal_ctx_sem_seg.py
- prepare_pascal_voc_sem_seg.py
- README.md
- demo.cpython-38.pyc
- demo.cpython-39.pyc
- ade.jpg
- coco.jpg
- ego4d.jpg
- image_edit.png
- image_gen.png
- purse.jpeg
- sample_0.png
- sample_1.png
- app.py
- demo.py
- gen_mask.py
- Dockerfile
- github_arch.gif
- github_vis_ade_0.gif
- github_vis_ade_1.gif
- github_vis_coco_0.gif
- github_vis_coco_1.gif
- github_vis_ego4d_0.gif
- github_vis_ego4d_1.gif
- teaser.jpg
- __init__.cpython-39.pyc
- __init__.cpython-39.pyc
- odise_checkpointer.cpython-39.pyc
- __init__.py
- odise_checkpointer.py
- __init__.cpython-39.pyc
- instantiate.cpython-39.pyc
- utils.cpython-39.pyc
- __init__.py
- instantiate.py
- utils.py
- __init__.cpython-39.pyc
- build.cpython-39.pyc
- dataset_mapper.cpython-39.pyc
- __init__.cpython-39.pyc
- register_coco_caption.cpython-39.pyc
- register_pascal.cpython-39.pyc
- ade20k_150.txt
- ade20k_150_with_prompt_eng.txt
- ade20k_847.txt
- ade20k_847_with_prompt_eng.txt
- coco_panoptic.txt
- coco_panoptic_with_prompt_eng.txt
- lvis_1203.txt
- lvis_1203_with_prompt_eng.txt
- pascal_context_459.txt
- pascal_context_459_with_prompt_eng.txt
- pascal_context_59.txt
- pascal_context_59_with_prompt_eng.txt
- pascal_voc_21.txt
- pascal_voc_21_with_prompt_eng.txt
- README.md
- __init__.py
- register_coco_caption.py
- register_pascal.py
- __init__.py
- build.py
- dataset_mapper.py
- __init__.cpython-39.pyc
- defaults.cpython-39.pyc
- train_loop.cpython-39.pyc
- __init__.py
- defaults.py
- hooks.py
- train_loop.py
- __init__.cpython-39.pyc
- d2_evaluator.cpython-39.pyc
- evaluator.cpython-39.pyc
- __init__.py
- d2_evaluator.py
- evaluator.py
- __init__.py
- configs
- model_zoo.py
- __init__.cpython-39.pyc
- preprocess.cpython-39.pyc
- __init__.cpython-39.pyc
- feature_extractor.cpython-39.pyc
- __init__.py
- feature_extractor.py
- __init__.cpython-39.pyc
- diffusion_builder.cpython-39.pyc
- gaussian_diffusion.cpython-39.pyc
- respace.cpython-39.pyc
- __init__.py
- diffusion_builder.py
- gaussian_diffusion.py
- resample.py
- respace.py
- __init__.cpython-39.pyc
- clip.cpython-39.pyc
- helper.cpython-39.pyc
- ldm.cpython-39.pyc
- odise.cpython-39.pyc
- __init__.py
- clip.py
- helper.py
- ldm.py
- odise.py
- __init__.cpython-39.pyc
- pano_wrapper.cpython-39.pyc
- __init__.py
- pano_wrapper.py
- __init__.py
- preprocess.py
- __init__.cpython-39.pyc
- collect_env.cpython-39.pyc
- file_io.cpython-39.pyc
- parameter_count.cpython-39.pyc
- __init__.py
- collect_env.py
- events.py
- file_io.py
- parameter_count.py
- __init__.py
- dependency_links.txt
- PKG-INFO
- requires.txt
- SOURCES.txt
- top_level.txt
- __init__.py
- coco_instance_new_baseline_dataset_mapper.py
- coco_panoptic_new_baseline_dataset_mapper.py
- mask_former_instance_dataset_mapper.py
- mask_former_panoptic_dataset_mapper.py
- mask_former_semantic_dataset_mapper.py
- __init__.py
- register_ade20k_full.py
- register_ade20k_instance.py
- register_ade20k_panoptic.py
- register_coco_panoptic_annos_semseg.py
- register_coco_stuff_10k.py
- register_mapillary_vistas.py
- register_mapillary_vistas_panoptic.py
- __init__.py
- __init__.py
- instance_evaluation.py
- __init__.py
- swin.py
- __init__.py
- mask_former_head.py
- per_pixel_baseline.py
- __init__.py
- ms_deform_attn_func.py
- __init__.py
- ms_deform_attn.py
- __init__.py
- test.py
- __init__.py
- fpn.py
- msdeformattn.py
- __init__.py
- mask2former_transformer_decoder.py
- maskformer_transformer_decoder.py
- position_encoding.py
- transformer.py
- __init__.py
- criterion.py
- matcher.py
- __init__.py
- misc.py
- __init__.py
- config.py
- maskformer_model.py
- test_time_augmentation.py
- __init__.py
- ytvos.py
- ytvoseval.py
- __init__.py
- builtin.py
- ytvis.py
- __init__.py
- augmentation.py
- build.py
- dataset_mapper.py
- ytvis_eval.py
- __init__.py
- position_encoding.py
- video_mask2former_transformer_decoder.py
- __init__.py
- criterion.py
- matcher.py
- __init__.py
- memory.py
- __init__.py
- config.py
- video_maskformer_model.py
- maskformer2_swin_large_IN21k_384_bs16_160k.yaml
- Base-ADE20K-InstanceSegmentation.yaml
- maskformer2_R50_bs16_160k.yaml
- maskformer2_swin_large_IN21k_384_bs16_160k.yaml
- Base-ADE20K-PanopticSegmentation.yaml
- maskformer2_R50_bs16_160k.yaml
- maskformer2_swin_base_384_bs16_160k_res640.yaml
- maskformer2_swin_base_IN21k_384_bs16_160k_res640.yaml
- maskformer2_swin_large_IN21k_384_bs16_160k_res640.yaml
- maskformer2_swin_small_bs16_160k.yaml
- maskformer2_swin_tiny_bs16_160k.yaml
- Base-ADE20K-SemanticSegmentation.yaml
- maskformer2_R101_bs16_90k.yaml
- maskformer2_R50_bs16_160k.yaml
- maskformer2_swin_base_IN21k_384_bs16_90k.yaml
- maskformer2_swin_large_IN21k_384_bs16_90k.yaml
- maskformer2_swin_small_bs16_90k.yaml
- maskformer2_swin_tiny_bs16_90k.yaml
- Base-Cityscapes-InstanceSegmentation.yaml
- maskformer2_R101_bs16_90k.yaml
- maskformer2_R50_bs16_90k.yaml
- maskformer2_swin_base_IN21k_384_bs16_90k.yaml
- maskformer2_swin_large_IN21k_384_bs16_90k.yaml
- maskformer2_swin_small_bs16_90k.yaml
- maskformer2_swin_tiny_bs16_90k.yaml
- Base-Cityscapes-PanopticSegmentation.yaml
- maskformer2_R101_bs16_90k.yaml
- maskformer2_R50_bs16_90k.yaml
- maskformer2_swin_base_IN21k_384_bs16_90k.yaml
- maskformer2_swin_large_IN21k_384_bs16_90k.yaml
- maskformer2_swin_small_bs16_90k.yaml
- maskformer2_swin_tiny_bs16_90k.yaml
- Base-Cityscapes-SemanticSegmentation.yaml
- maskformer2_R101_bs16_90k.yaml
- maskformer2_R50_bs16_90k.yaml
- maskformer2_swin_base_384_bs16_50ep.yaml
- maskformer2_swin_base_IN21k_384_bs16_50ep.yaml
- maskformer2_swin_large_IN21k_384_bs16_100ep.yaml
- maskformer2_swin_small_bs16_50ep.yaml
- maskformer2_swin_tiny_bs16_50ep.yaml
- Base-COCO-InstanceSegmentation.yaml
- maskformer2_R101_bs16_50ep.yaml
- maskformer2_R50_bs16_50ep.yaml
- maskformer2_swin_base_384_bs16_50ep.yaml
- maskformer2_swin_base_IN21k_384_bs16_50ep.yaml
- maskformer2_swin_large_IN21k_384_bs16_100ep.yaml
- maskformer2_swin_small_bs16_50ep.yaml
- maskformer2_swin_tiny_bs16_50ep.yaml
- Base-COCO-PanopticSegmentation.yaml
- maskformer2_R101_bs16_50ep.yaml
- maskformer2_R50_bs16_50ep.yaml
- maskformer2_swin_large_IN21k_384_bs16_300k.yaml
- Base-MapillaryVistas-PanopticSegmentation.yaml
- maskformer_R50_bs16_300k.yaml
- maskformer2_swin_large_IN21k_384_bs16_300k.yaml
- Base-MapillaryVistas-SemanticSegmentation.yaml
- maskformer2_R50_bs16_300k.yaml
- video_maskformer2_swin_base_IN21k_384_bs16_8ep.yaml
- video_maskformer2_swin_large_IN21k_384_bs16_8ep.yaml
- video_maskformer2_swin_small_bs16_8ep.yaml
- video_maskformer2_swin_tiny_bs16_8ep.yaml
- Base-YouTubeVIS-VideoInstanceSegmentation.yaml
- video_maskformer2_R101_bs16_8ep.yaml
- video_maskformer2_R50_bs16_8ep.yaml
- video_maskformer2_swin_base_IN21k_384_bs16_8ep.yaml
- video_maskformer2_swin_large_IN21k_384_bs16_8ep.yaml
- video_maskformer2_swin_small_bs16_8ep.yaml
- video_maskformer2_swin_tiny_bs16_8ep.yaml
- Base-YouTubeVIS-VideoInstanceSegmentation.yaml
- video_maskformer2_R101_bs16_8ep.yaml
- video_maskformer2_R50_bs16_8ep.yaml
- ade20k_instance_catid_mapping.txt
- ade20k_instance_imgCatIds.json
- prepare_ade20k_ins_seg.py
- prepare_ade20k_pan_seg.py
- prepare_ade20k_sem_seg.py
- prepare_coco_semantic_annos_from_panoptic_annos.py
- README.md
- demo.py
- predictor.py
- README.md
- demo.py
- predictor.py
- README.md
- visualizer.py
- __init__.cpython-39.pyc
- config.cpython-39.pyc
- maskformer_model.cpython-39.pyc
- test_time_augmentation.cpython-39.pyc
- __init__.cpython-39.pyc
- __init__.cpython-39.pyc
- coco_instance_new_baseline_dataset_mapper.cpython-39.pyc
- coco_panoptic_new_baseline_dataset_mapper.cpython-39.pyc
- mask_former_instance_dataset_mapper.cpython-39.pyc
- mask_former_panoptic_dataset_mapper.cpython-39.pyc
- mask_former_semantic_dataset_mapper.cpython-39.pyc
- __init__.py
- coco_instance_new_baseline_dataset_mapper.py
- coco_panoptic_new_baseline_dataset_mapper.py
- mask_former_instance_dataset_mapper.py
- mask_former_panoptic_dataset_mapper.py
- mask_former_semantic_dataset_mapper.py
- __init__.cpython-39.pyc
- register_ade20k_full.cpython-39.pyc
- register_ade20k_instance.cpython-39.pyc
- register_ade20k_panoptic.cpython-39.pyc
- register_coco_panoptic_annos_semseg.cpython-39.pyc
- register_coco_stuff_10k.cpython-39.pyc
- register_mapillary_vistas.cpython-39.pyc
- register_mapillary_vistas_panoptic.cpython-39.pyc
- __init__.py
- register_ade20k_full.py
- register_ade20k_instance.py
- register_ade20k_panoptic.py
- register_coco_panoptic_annos_semseg.py
- register_coco_stuff_10k.py
- register_mapillary_vistas.py
- register_mapillary_vistas_panoptic.py
- __init__.py
- __init__.cpython-39.pyc
- instance_evaluation.cpython-39.pyc
- __init__.py
- instance_evaluation.py
- __init__.cpython-39.pyc
- criterion.cpython-39.pyc
- matcher.cpython-39.pyc
- __init__.cpython-39.pyc
- swin.cpython-39.pyc
- __init__.py
- swin.py
- __init__.cpython-39.pyc
- mask_former_head.cpython-39.pyc
- per_pixel_baseline.cpython-39.pyc
- __init__.py
- mask_former_head.py
- per_pixel_baseline.py
- __init__.cpython-39.pyc
- fpn.cpython-39.pyc
- msdeformattn.cpython-39.pyc
- __init__.cpython-39.pyc
- __init__.cpython-39.pyc
- ms_deform_attn_func.cpython-39.pyc
- __init__.py
- ms_deform_attn_func.py
- __init__.cpython-39.pyc
- ms_deform_attn.cpython-39.pyc
- __init__.py
- ms_deform_attn.py
- ms_deform_attn_cpu.cpp
- ms_deform_attn_cpu.h
- ms_deform_attn_cpu.o
- ms_deform_attn_cuda.cu
- ms_deform_attn_cuda.h
- ms_deform_attn_cuda.o
- ms_deform_im2col_cuda.cuh
- ms_deform_attn.h
- vision.cpp
- vision.o
- __init__.py
- test.py
- __init__.py
- fpn.py
- msdeformattn.py
- __init__.cpython-39.pyc
- mask2former_transformer_decoder.cpython-39.pyc
- maskformer_transformer_decoder.cpython-39.pyc
- position_encoding.cpython-39.pyc
- transformer.cpython-39.pyc
- __init__.py
- mask2former_transformer_decoder.py
- maskformer_transformer_decoder.py
- position_encoding.py
- transformer.py
- __init__.py
- criterion.py
- matcher.py
- __init__.cpython-39.pyc
- misc.cpython-39.pyc
- __init__.py
- misc.py
- __init__.py
- config.py
- maskformer_model.py
- test_time_augmentation.py
- dependency_links.txt
- PKG-INFO
- requires.txt
- SOURCES.txt
- top_level.txt
- __init__.py
- ytvos.py
- ytvoseval.py
- __init__.py
- builtin.py
- ytvis.py
- __init__.py
- augmentation.py
- build.py
- dataset_mapper.py
- ytvis_eval.py
- __init__.py
- position_encoding.py
- video_mask2former_transformer_decoder.py
- __init__.py
- criterion.py
- matcher.py
- __init__.py
- memory.py
- __init__.py
- config.py
- video_maskformer_model.py
- analyze_model.py
- convert-pretrained-swin-model-to-d2.py
- convert-torchvision-to-d2.py
- evaluate_coco_boundary_ap.py
- evaluate_pq_for_semantic_segmentation.py
- README.md
- ADVANCED_USAGE.md
- CODE_OF_CONDUCT.md
- cog.yaml
- CONTRIBUTING.md
- GETTING_STARTED.md
- INSTALL.md
- LICENSE
- MODEL_ZOO.md
- predict.py
- README.md
- requirements.txt
- setup.py
- train_net.py
- train_net_video.py
- train_net.py
- setup.cfg
- setup.py
- __init__.cpython-38.pyc
- __init__.cpython-39.pyc
- base_options.cpython-38.pyc
- base_options.cpython-39.pyc
- test_options.cpython-38.pyc
- test_options.cpython-39.pyc
- train_options.cpython-38.pyc
- train_options.cpython-39.pyc
- __init__.py
- base_options.py
- test_options.py
- train_options.py
- __init__.cpython-38.pyc
- __init__.cpython-39.pyc
- flow_to_color.cpython-38.pyc
- flow_to_color.cpython-39.pyc
- html.cpython-38.pyc
- html.cpython-39.pyc
- image_pool.cpython-38.pyc
- image_pool.cpython-39.pyc
- util.cpython-38.pyc
- util.cpython-39.pyc
- visualizer.cpython-38.pyc
- visualizer.cpython-39.pyc
- __init__.py
- flow_to_color.py
- html.py
- image_pool.py
- util.py
- visualizer.py
- inference_t2c.py
- LICENSE
- README.md
- requirements.txt
- test_motion.py
- test_motion_directional.py
- test_video.py
- train_motion.py
- train_video.py
# 설치 가이드
1. 코드 내려받기
git clone https://github.com/text2cinemagraph/text2cinemagraph
깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.
cd text2cinemagraph
방금 내려받은 프로젝트 폴더 안으로 이동합니다.
2. 공식 설치 스크립트
쉬움 추천사전 준비물
- Python 3 pip 명령어를 쓰려면 Python이 필요합니다.
pip install git+https://github.com/NVlabs/ODISE.git
PyPI에 배포된 패키지를 바로 설치합니다. 소스 클론이 필요 없습니다.
설치 후 새 터미널을 열고, 프로그램의 버전 확인 명령(예: --version)으로 정상 설치됐는지 확인하세요.
이 레포의 README에 적힌 실제 명령어를 그대로 가져왔습니다.
3. Docker
쉬움사전 준비물
- Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
- Docker Desktop 컨테이너를 빌드하고 실행하려면 필요합니다. 설치 후 실행해서 백그라운드에 켜두세요.
⚠️ 이 프로젝트는 규모가 큰 저장소라, 이 방법이 실제 핵심 제품이 아니라 내부 하위 패키지를 가리키는 것일 수 있습니다. README 전체를 함께 확인해보세요.
docker build -f ODISE/docker/Dockerfile -t text2cinemagraph .
Dockerfile을 기반으로 실행 가능한 이미지를 빌드합니다.
docker run -p 8080:80 text2cinemagraph
빌드된 이미지를 실제 컨테이너로 실행합니다.
터미널에 docker compose ps 를 입력해 컨테이너들이 Up 상태인지 확인하세요. README에 포트 번호가 적혀있다면 브라우저에서 http://localhost:포트번호 로 접속해보세요.
4. Python
쉬움사전 준비물
pip install git+https://github.com/NVlabs/ODISE.git
PyPI에 배포된 패키지를 바로 설치합니다. 소스 클론이 필요 없습니다.
pip install -r requirements.txt
requirements.txt 등에 명시된 파이썬 라이브러리를 설치합니다.
에러 메시지 없이 실행되고 터미널에 안내 문구가 출력되면 정상입니다.
이 레포의 README에 적힌 실제 명령어를 그대로 가져왔습니다.
// repository documentation
Was this content helpful?
(0 ratings)