360Cam-PGM-3DGS-Tools

KO | EN

gitlite — search

#python #typescript #go #golang #android #machine-learning #deep-learning #docker #java #awesome-list #vue #mcp #openai #open-source

360Cam-PGM-3DGS-Tools

(★ 57)

A toolkit for turning 360° video captures into photogrammetry (PGM) datasets or 3D Gaussian Splatting (3DGS) datasets.

360Cam-PGM-3DGS-Tools 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# 360Cam-PGM-3DGS-Tools

A consolidated toolkit for turning 360° video captures into photogrammetry (PGM) datasets and optimized point clouds that seed 3D Gaussian Splatting (3DGS) projects.
The scripts cover every stage from frame extraction to PLY refinement, and the desktop GUI orchestrates the full flow for multiple photogrammetry and 3DGS apps.

> **Project status:** actively edited and expanded. Expect interface polish, richer presets, and additional documentation updates in upcoming revisions.
> The code in this repository was generated with the assistance of OpenAI Codex.

---

## Installation & Environment Setup

### Requirements
- Python **3.7** or newer
- `pip` for installing Python packages
- Python dependencies listed in `requirements.txt`
- [FFmpeg](https://ffmpeg.org/) available on your `PATH`
- PyTorch (`torch` **1.10+** and matching `torchvision` build) for the human masking tool
- GPU/CUDA is optional for masking, but accelerates large batches.

### Setup Steps
1. **Clone the repository**
   ```bash
   git clone https://github.com/Mistral-Yu/360Cam-PGM-3DGS-Tools.git
   cd 360Cam-PGM-3DGS-Tools
   ```
2. **Create and activate a Conda environment**
   ```bash
   conda create -n gs360 python=3.7
   conda activate gs360
   ```
   > **Tip:** pick any supported Python version (3.7+) that matches your GPU drivers/CUDA toolkits.
3. **Install Python dependencies**
   ```bash
   pip install -r requirements.txt
   ```
   > **Note:** PyTorch wheels are platform-specific. If `pip` cannot find a build for your OS/Python combo, install `torch`/`torchvision` manually inside the Conda environment before rerunning the command.
4. **Verify FFmpeg**
   ```bash
   ffmpeg -version
   ```

---

## Workflow at a Glance
GUI Explanation: `gs360_360GUI.py` (Tabs: `Video2Frames`, `FrameSelector`, `360PerspCut`, `SegmentationMaskTool`, `PointCloudOptimizer`, `MS360xmlToPerspCams`).
- `Video2Frames` tab: Sample equirectangular frames from a 360° video.
- `FrameSelector` tab: Score and retain the sharpest frames for Structure-from-Motion (SfM).
- `360PerspCut` tab: Convert panoramas into perspective views.
- `SegmentationMaskTool` tab: Use SegmentationMaskTool to preview and refine masks for unwanted subjects or rig elements before reconstruction to reduce ghosts and cleanup artifacts.
- Use exported views to align in your photogrammetry software (RealityScan, Metashape, etc.) and export the PLY point cloud plus camera metadata.
- `PointCloudOptimizer` tab: Optimize initial point cloud, Downsample/merge the PLY or Colmap PointCloud3D for your 3DGS tool (PostShot, gsplat, etc.) . Delete sky point clouds generated with photogrammetry software, add clean Sky point clouds.
- `MS360xmlToPerspCams` tab: Experimental tool for convert Metashape spherical camera XML into perspective camera parameters. Colmap, Metashape xml, transform.json, RealityScan xmp.
- `DualFisheyePipeline` tab: Experimental tool for correcting dual-fisheye distortion from Metashape XML calibration and exporting perspective views.
- `CameraOptimization` tab: Experimental tool. Preview and convert camera poses and point clouds between supported formats. TODO: Reduce camera count to lower VRAM/RAM usage.

### Workflows (GUI)
https://mistral-yu.github.io/360Cam-3DGS-Samples/

#### Rapid Workflow
1. Create a 360 video using DJI Studio or similar software (This GitHub repository has been tested with the Osmo 360. Recommended settings: apply D-Log only, keep all other settings at their default value of 0 if correct color values are required under a linear workflow. And turn RockSteady off.)
2. Launch `gs360_360GUI.py`.
3. In the `360PerspCut` tab, click **Browse Video** and select a video file.
4. Choose a preset.
5. Under **Video (direct export)**, set the FPS.
6. Click **Run Export** to write the images.
7. **RealityScan**: launch the app, import images, then select all.
   Set Prior Calibration -> Prior to **Fixed** or Prior to Approximate, and change **Focal Length** to the value shown in the `360PerspCut` log
   (Preset default: 12 mm, fisheyelike: 17mm, full360coverage: 14 mm).
   Set Prior Lens Distortion -> Prior to **Fixed**.
8. Or **Metashape**: launch the app, import images, then go to Tools -> **Camera Calibration (Initial tab)**.
   Set Type to **Precalibration** and update **f** to the value shown in the `360PerspCut` log
   (Preset default: 533.33333, fisheyelike: 755.55556, full360coverage: 622.22222).
   Next, click the **Fixed parameters** Select button and **Check all** or check distortion parameters except f.
9. Bring the RealityScan or Metashape alignment results into a 3DGS tool such as PostShot.

#### Faster Workflow but low quality
1. Create a 360 video using DJI Studio or similar software.
2. Launch `gs360_360GUI.py`.
3. In the `Video2Frames` tab, extract 360 frames by setting the FPS and running the export.
4. In **Metashape**, import the frames, then go to **Tools -> Camera Calibration** and set **Camera Type = Spherical**.
5. Align the cameras.
6. Export from Metashape: **File -> Export -> Export Cameras** (Agisoft XML) and **File -> Export -> Point Cloud** (PLY).
7. Back in `gs360_360GUI.py`, open the **MS360xmlToPerspCams** tab and set **Input XML** to the exported XML.
8. Set **Format = transforms**, enable **PerspCut**, set **PerspCut input** to the 360 image folder, and set **Points PLY** to the exported PLY. Run the tool.
9. Use **PerspCut out**, `transforms.json`, and the rotated PLY in PostShot (or similar 3DGS tools).

### Recommended Workflow 
- Metashape Pro, Multi-Cameras-System.  (**Known bug**: If you import perspective_cams_Multi-Camera-System.xml with MS360xmlToPerspCams and no tie points appear after alignment(sometimes), re-importing the XML file fixes the issue.).
- RealityCapture or Metashape Standard with spherical Metashape Alignment.xml.

**NOTES**
- After converting to the RealityScan format (XMP) using **MS360xmlToPersCams**, be careful with the **Position Accuracy** setting in the alignment settings when running alignment in RealityScan.
The default value is too large — it is intended for aerial capture.

---

## License

---

## Work in progress
- Dual fisheye extraction → Undistort, apply D-log M to sRGB, Correct color differents → Convert to 5 perspective images → Update Metashape Multi-Camera System workflow
- CameraFormatConverter. RealityScan(CSV, Ply, XMP) ←→ Metashape(XML, Ply) ←→ Colmap

## TODO / Ideas
- Flesh out a full GUI walkthrough (tab descriptions, launch parameters, screenshot gallery).

360Cam-PGM-3DGS-Tools

360Cam-PGM-3DGS-Tools 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

360Cam-PGM-3DGS-Tools

360Cam-PGM-3DGS-Tools 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README