4Distillation

KO | EN

gitlite — search

#javascript #hacktoberfest #llm #golang #linux #docker #nodejs #java #awesome #awesome-list #vue #security #openai #open-source

(★ 11)

Joint Time and View Control via 4D-to-2D Knowledge Distillation

4Distillation 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h1 align="center">4Distillation</h1>
<h3 align="center">Joint Time and View Control via 4D-to-2D Knowledge Distillation <br>leveraging Spatiotemporal Implicit Representations</h3>
<p align="center">
  <img src="https://github.com/eviatar-ben/4Distillation/blob/main/assets/figures/overview.png?raw=true" alt="4Distillation Overview" width="80%">
</p>

<p align="center" style="font-style: italic; font-size: 0.9em;">
Generated 4D scene example using the L4GM reconstruction model
</p>

This repository presents **4Distillation**, a novel method for distilling knowledge from ***4D Reconstruction model*** into a ***Generative 2D diffusion model***.  
Our approach enables **joint control over time and viewpoint** in a **generalized feedforward manner**.

**Status**: 🚧 This project is a work in progress - expect breaking changes.

---

## Method

We introduce a distillation-based framework that transfers temporal and viewpoint understanding from 4D data into a 2D generative model (e.g., Stable Diffusion).

The overview in the figure focuses on the temporal dimension, but the viewpoint dimension is learned in a fully analogous manner using a View Mapper instead of a Time Mapper, and conditioning on the relative camera pose difference instead of the frame index difference.

---

### Step 1: Sampling Temporal & Viewpoints Pairs from Videos & Scenes

---

### Step 2: Time Mapper — Encoding Motion

Following the NeTI approach, to maintain the editability–fidelity trade-off and to constrain **v_bypass** from becoming too dominant, the textual condition **v\*** is calculated as:

---

### Step 3: Temporal & View Knowledge Distillation into the Time & View Mappers via the 2D Diffusion Model

---
## Acknowledgments

This project builds upon and incorporates methods from the following works:

- [NeTI](https://arxiv.org/abs/2305.15391)
- [ViewNeTI](https://arxiv.org/abs/2309.07986)
- [IPAdapter](https://arxiv.org/abs/2308.06721)

- [IP-Adapter by Tencent AI Lab](https://github.com/tencent-ailab/IP-Adapter)  
- [ComfyUI IPAdapter Plus by cubiq](https://github.com/cubiq/ComfyUI_IPAdapter_plus)

We gratefully acknowledge the original authors for their contributions.  
All original code, models, and methods remain the property of their respective creators.

Copyright Notice:
This project utilizes methods, code, and ideas from various sources, including NeTI, ViewNeTI, IPAdapter, and others.
All rights to the original methods and code remain with their respective owners.
Some sources or authors may not be explicitly mentioned here. If any copyright concerns arise, please contact me so I can address them promptly.

4Distillation

4Distillation 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

4Distillation

4Distillation 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README