KO
|
EN
gitlite — search
Search
#nextjs
#typescript
#javascript
#python
#home-assistant
#ai
#java
#html
#tailwindcss
#react
#discord
#nextjs14
VirtualObjectInsertion
★ 54
Open GitHub ↗
No description available.
Download README (.md)
Explore Similar Repositories
Keras-Tuner
:
No description available.
wechat-ipad
:
基于c#的 微信ipad协议，源码全部开源，领提供golang版本的
glitching_images
:
A small compilation of scripts and trivia related to glitching images. Based on the article on https://venam.nixers.net/blog/programming/2020/10/05/corruption-at-the-core.html
hire
:
Working with intelligent people.
registry
:
注册中心
// repository documentation
Was this content helpful?
★ 0
(0 ratings)
Select Rating:
★
★
★
★
★
Submit Feedback
Recent Feedback
×
Download README
Do you want to download the
README.md
file for
VirtualObjectInsertion
?
Download (.md)
# Virtual Object Insertion This repository contains the code to generate the object insertion results in paper [Inverse rendering for complex indoor scenes: Shape, spatially-varying lighting and svbrdf from a single image, CVPR 2020](https://drive.google.com/file/d/18zG1kzVpL9XsEVBK95hbpnB-FMlChRXP/view). Please visit our [project webpage](http://cseweb.ucsd.edu/~viscomp/projects/CVPR20InverseIndoor/) for more information. We hope that the code can help reproduce the photorealistic object insertion results in our paper and compare future lighting estimation methods more easily. Please consider citing the paper if you find the code useful. Please contact [Zhengqin Li](zhl378@eng.ucsd.edu) if you have any questions. ## Results and Comparisons The original models were trained by extending the SUNCG dataset with an SVBRDF-mapping. Since SUNCG is not available now due to copyright issues, we are not able to release the original models. Instead, we rebuilt a new high-quality synthetic indoor scene dataset and trained our models on it. We will release the new dataset in the near future. Some object insertion results on real images generated using our network trained on the new dataset are shown below.  The quantitative and qualitative comparisons with prior works on object insertion are shown below.   Compared with our networks trained on SUNCG-based dataset, our networks trained on new dataset can predict more consistent lighting color and intensity. Therefore, it achieves better performance in user study. ## Prerequisites To run our code, you will need * matlab * python * Renderer: Please find the renderer from this [link](https://github.com/lzqsd/OptixRenderer), which is an Optix-based renderer implemented for this project. ## Data Please download the data from the [link](http://cseweb.ucsd.edu/~viscomp/projects/CVPR20InverseIndoor/objectInsertion.zip). It includes the inverse rendering predictions of the 20 examples from Garon et al. 2019 dataset (Example 1-20) and 4 examples (demo0-3) downloaded from the Internet. ## Instructions To reproduce the 20 object insertion results from Garon et al. 2019 dataset. * Download the inverse rendering predictions from the [link](http://cseweb.ucsd.edu/~viscomp/projects/CVPR20InverseIndoor/objectInsertion_test.zip) mentioned above. Unzip the data in the same directory as the code. * Open `generateXMLread_ref.m`. On line 3 and 4, write down the path to your renderer and your python program. * Run `generateXMLread_ref.m` The program will read the `im_*.mat` files in each `Example*` directory to generate the object insertion results. We also provide the code to generate the object insertion code from scratch. The instructions are as follows: * Similarly, download the inverse rendering results from this [link](http://cseweb.ucsd.edu/~viscomp/projects/CVPR20InverseIndoor/objectInsertion_test.zip) * Open `generateXML.m`. On line 4 and 5, enter the path to your renderer and your python program. * Several parameters from can be used to adjust the object insertion results * `meshNewName`: The mesh used for object insertion. We provide two meshes: `sphere.obj` and `bunny.obj`. You can include your own mesh but you may need to scale and rotate the mesh accordingly so that they will look good. * `r, g, b`: The diffuse color of the inserted object. We set them all to 0.8 in our experiments. * `roughness`: The roughness value of the inserted object. The smaller the roughness value, the more specular the object will be. We set roughness value to 0.2 in all our experiments. * `scale`: The size of the inserted object. The size of the inserted object is decided by the size of the plane you selected from the image (will be explained more detailedly in the following) and the scale parameter. * `isNewMask`: Whether to select new plane or not. Sometimes, after selecting the plane, you may find the rendered object too small or too large. In that case, you can set `isNewMask` to be `false` and adjust `scale`, until you get the satisfactory object insertion result. * Run `generateXML.m`: * You will see an image poped out. Choose 4 points on the image as the plane you select. An example is shown below.  * After choosing the 4 point, the image will be shown again with a red square polygon demonstrating the plane you selected. Choose a point inside the polygon as the place where you would like to inser the object.  * Done, the rendered result is shown below. The image will be saved as `im_new.png` in the `root` directory set on line 3.  * If you woild like to insert more objects, just change the `originName` from `im.png` to `im_new.png`. Repeat the process and you can insert multiple objects at different locations of a single image. ## Limitations The current code only supports inserting an object on a planar surface. It has not been extended to handle objects in arbitrary 3D geometry yet.
