sscnet

KO | EN

gitlite — search

#python #ai #golang #go #machine-learning #rust #deep-learning #docker #macos #awesome #kubernetes #awesome-list #pytorch #openai

(★ 347)

Semantic Scene Completion from a Single Depth Image

File Explorer

Download Latest Version (.zip)

- - .Doxyfile
  - .gitignore
  - .travis.yml
  - caffe.cloc
  - CMakeLists.txt
  - CONTRIBUTING.md
  - CONTRIBUTORS.md
  - INSTALL.md
  - LICENSE
  - Makefile
  - Makefile.config.example
  - Makefile.config.sscnet_example
  - README.md
- demo.ply
- demo.txt
- demotest_model.py
- NYU0001_0000.bin
- NYU0001_0000.png
- teaser.jpg
- - NYUclassMapping40.mat
  - SUNCG_test_sceneId.mat
  - SUNCG_train_sceneId.mat
  - test_NYUv2Ids.mat
  - test_NYUv2images.mat
  - train_NYUv2Ids.mat
  - train_NYUv2images.mat
  - voxletTest.mat
- - demo.m
  - model.mat
  - plot3D.m
  - polygon2voxel.m
  - polygon2voxel_double.c
  - polygon2voxel_double.mexa64
  - polygon2voxel_double.mexmaci64
  - sparse3D.m
  - voxel.m
- - renderDepth
  - renderDepth.cpp
  - renderMesh.hpp
- - camExtrinsics2Pose.m
  - camPose2Extrinsics.m
  - colorfourty.m
  - downSample.m
  - extrinsics2camPose.m
  - file2string.m
  - getcolorPalette.m
  - getobjclassSUNCG.m
  - loadjson.m
  - mesh2off.m
  - patch2ply.m
  - points2normals.m
  - quaternion.m
  - read_3d_pts_general.m
  - read_binvox.m
  - read_wobj.m
  - readCameraName.m
  - readDepth.m
  - readRLEfile.m
  - rectify.m
  - saveDepth.m
  - show3DModel.m
  - show_volume.m
  - struct2xml.m
  - volume2voxelmesh.m
  - write_binvox.m
  - writeRLEfile.m
  - xml2struct.m
- ClassMapping.mat
- evaluate_completion.m
- evaluate_prediction.m
- evaluation_script.m
- genSUNCGdataScript.m
- getSceneVoxSUNCG.m
- perpareDataTest.m
- perpareNYUCADdata.m
- readCameraPose.m
- renderView.m
- suncgObjcategory.mat
- volume_param.m
- demo.txt
- demotest_model.py
- NYU0001_0000.bin
- NYU0001_0000.png
.gitignore
download_data.sh
download_models.sh
download_objectvox.sh
download_results.sh
download_suncgTrain.sh
download_UIUCCAD.sh
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Semantic Scene Completion from a Single Depth Image

This repo contains training and testing code for our paper on semantic scene completion, a task for producing a complete 3D voxel representation of volumetric occupancy and semantic labels for a scene from a single-view depth map observation. 
More information about the project can be found in our [paper](https://arxiv.org/pdf/1611.08974v1.pdf) and [project webset](http://sscnet.cs.princeton.edu/)

![teaser](https://raw.githubusercontent.com/shurans/sscnet/master/image/teaser.jpg)

If you find [SSCNet](http://sscnet.cs.princeton.edu/) useful in your research, please cite:

@article{song2016ssc,
      author     = {Song, Shuran and Yu, Fisher  and Zeng, Andy and Chang, Angel X and Savva, Manolis and Funkhouser, Thomas},
      title      = {Semantic Scene Completion from a Single Depth Image},
      journal    = {arXiv preprint arXiv:1611.08974},
      year       = {2016},
    }

### Contents
0. [Organization](#organization)
0. [Installation](#installation)
0. [Quick Demo](#quick-demo)
0. [Testing](#testing)
0. [Training](#training)
0. [Visualization and Evaluation](#visualization-and-evaluation)
0. [Data Preparation](#data)

### Installation
0. Software Requirements: 
   1. Requirements for `Caffe` and `pycaffe` (see: [Caffe installation instructions](http://caffe.berkeleyvision.org/installation.html))
   2. Matlab 2016a or above with vision toolbox 
   3. OPENCV
0. Hardware Requirements:  at least 12G GPU memory.
0. Install caffe and pycaffe. 
    1. Modify the config files based on your system. You can reference Makefile.config.sscnet_example.
    2. Compile  
    ```Shell
    cd caffe_code/caffe3d_suncg
    # Now follow the Caffe installation instructions here:
    #   http://caffe.berkeleyvision.org/installation.html
    make -j8 && make pycaffe
    ```
0. Export path

```Shell 
     export LD_LIBRARY_PATH=~/build_master_release/lib:/usr/local/cudnn/v5/lib64:~/anaconda2/lib:$LD_LIBRARY_PATH
     export PYTHONPATH=~/build_master_release/python:$PYTHONPATH
     ```

### Quick Demo:
  ```Shell 
  cd demo
  python demotest_model.py
  ```
This demo runs semantic scene compeletion on one NYU depth map using our pretrained model and outputs a '.ply' visulization of the result.

### Testing:
0. Run the testing script
  ` cd caffe_code/script/test
    python test_model.py`
0. The output results will be stored in folder `results` in .hdf5 format
0. To test on other testsets (e.g. suncg, nyu, nyucad) you need to modify the paths in “test_model.py”.

### Training:
0. Finetuning on NYU 
    `cd caffe_code/train/ftnyu
      ./train.sh`
0. Training from scratch 
    ` cd caffe_code/train/trainsuncg
    ./train.sh`
0. To get more training data from SUNCG, please refer to the SUNCG toolbox

### Visualization and Evaluation:
0. After testing, the results should be stored in folder `results/`
0. You can also download our precomputed results:
   `./download_results.sh`
0. Run the evaluation code in matlab:

``` shell 
    matlab &
    cd matlab_code
    evaluation_script('../results/','nyucad')
    ```
0. The visualization of results will be stored in `results/nyucad` as “.ply” files.

### Data 
0. Data format 
    1. Depth map : 
        16 bit png with bit shifting.
        Please refer to `./matlab_code/utils/readDepth.m` for more information about the depth format.
    2. 3D volume: 
        First three float stores the origin of the 3D volume in world coordinate.
        Then 16 float of camera pose in world coordinate.
        Followed by the 3D volume encoded by run-length encoding.
        Please refer to `./matlab_code/utils/readRLEfile.m` for more details.
0. Example code to convert NYU ground truth data: `matlab_code/perpareNYUCADdata.m` 
   This function provides an example of how to convert the NYU ground truth from 3D CAD model annotations provided by:
   Guo, Ruiqi, Chuhang Zou, and Derek Hoiem. "Predicting complete 3d models of indoor scenes."
   You need to download the original annotations by runing `download_UIUCCAD.sh`.  
0. Example code to generate testing data without ground truth and room boundary: `matlab_code/perpareDataTest.m`
   This function provides an example of how to generate your own testing data without ground truth labels. It will generate a the .bin file with camera pose and an empty volume, without room boundary.

### Generating training data from SUNCG

You can generate more training data from SUNCG by following steps:

0. Download SUNCG data and toolbox from: https://github.com/shurans/SUNCGtoolbox
0. Compile the toolbox.
0. Download the voxel data for objects (`download_objectvox.sh`) and move the folder under SUNCG data directory. 
0. Run the script: genSUNCGdataScript()
   You may need to modify the following paths:`suncgDataPath`, `SUNCGtoolboxPath`, `outputdir`.

### License
Code is released under the MIT License (refer to the LICENSE file for details).

sscnet

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

sscnet

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Download README