vk_lod_clusters

KO | EN

gitlite — search

#javascript #python #typescript #llm #golang #android #deep-learning #linux #awesome #cli #kubernetes #vue #security #ios

vk_lod_clusters

(★ 190)

Sample for cluster-based continuous level of detail rasterization or ray tracing

파일 탐색기

최종 버전 다운로드 (.zip)

- FindNvproCore2.cmake
- blas_caching.md
- blas_caching_benefit.jpg
- blas_caching_device.png
- blas_caching_host.png
- blas_caching_instances.png
- blas_merging.md
- blas_merging_benefit.jpg
- blas_merging_clusters.png
- blas_merging_implementation.png
- blas_merging_instances.png
- blas_sharing.md
- blas_sharing_adjustments.png
- blas_sharing_benefit.jpg
- blas_sharing_choice.png
- blas_sharing_example.png
- blas_sharing_implementation.png
- blas_sharing_lodrange.png
- blas_sharing_orientation.png
- blas_sharing_results.png
- blas_sharing_scene.jpg
- blas_sharing_traversal.png
- blas_techniques.png
- clas_allocation.md
- continuous_lod_clusters.png
- lod_allocation.png
- lod_arcsin_angular_error.svg
- lod_dag.svg
- lod_generate.jpg
- lod_generation.md
- lod_graph_cut.svg
- lod_hierarchy_selection.svg
- lod_rendering.png
- lod_spatial_selection.svg
- lod_stitch.svg
- lod_streaming.png
- lod_streaming_rt.png
- mirror_box.jpg
- otherscenes.jpg
- path_tracing.jpg
- streaming.md
- zorah_scene.jpg
- attribute_encoding.h
- blas_caching_setup_build.comp.glsl
- blas_caching_setup_copy.comp.glsl
- blas_clusters_insert.comp.glsl
- blas_setup_insertion.comp.glsl
- build_setup.comp.glsl
- culling.glsl
- dlss_util.h
- fullscreen.vert.glsl
- fullscreen_background.frag.glsl
- fullscreen_raster_atomic.frag.glsl
- fullscreen_write_depth.frag.glsl
- geometry_blas_sharing.comp.glsl
- hbao.h
- hbao_apply.comp.glsl
- hbao_calc.comp.glsl
- hbao_deinterleave.comp.glsl
- hbao_viewnormal.comp.glsl
- instance_assign_blas.comp.glsl
- instance_classify_lod.comp.glsl
- nvhiz-update.comp.glsl
- render_cluster_bbox.frag.glsl
- render_cluster_bbox.mesh.glsl
- render_instance_bbox.frag.glsl
- render_instance_bbox.mesh.glsl
- render_pathtrace.rgen.glsl
- render_pathtrace.rmiss.glsl
- render_pathtrace_clusters.rahit.glsl
- render_pathtrace_clusters.rchit.glsl
- render_pathtrace_hit.glsl
- render_raster.frag.glsl
- render_raster_clusters.mesh.glsl
- render_raster_clusters_sw.comp.glsl
- render_raytrace.rgen.glsl
- render_raytrace.rmiss.glsl
- render_raytrace_clusters.rahit.glsl
- render_raytrace_clusters.rchit.glsl
- render_shading.glsl
- shaderio.h
- shaderio_building.h
- shaderio_core.h
- shaderio_scene.h
- shaderio_streaming.h
- stream_agefilter_groups.comp.glsl
- stream_allocator_build_freegaps.comp.glsl
- stream_allocator_freegaps_insert.comp.glsl
- stream_allocator_load_groups.comp.glsl
- stream_allocator_setup_insertion.comp.glsl
- stream_allocator_unload_groups.comp.glsl
- stream_compaction_new_clas.comp.glsl
- stream_compaction_old_clas.comp.glsl
- stream_setup.comp.glsl
- stream_update_clas_geometry_indices.comp.glsl
- stream_update_scene.comp.glsl
- streaming.glsl
- texturing.glsl
- traversal.glsl
- traversal_blas_merging.comp.glsl
- traversal_init.comp.glsl
- traversal_init_blas_sharing.comp.glsl
- traversal_presort.comp.glsl
- traversal_run.comp.glsl
- traversal_run_groups.comp.glsl
- async_uploader.cpp
- async_uploader.hpp
- camera_path.cpp
- camera_path.hpp
- cgltf.cpp
- dlss_denoiser.cpp
- dlss_denoiser.hpp
- dlss_upscaler.cpp
- dlss_upscaler.hpp
- dlss_wrapper.cpp
- dlss_wrapper.hpp
- hbao_pass.cpp
- hbao_pass.hpp
- lodclusters.cpp
- lodclusters.hpp
- lodclusters_ui.cpp
- main.cpp
- meshopt_clusterlod.cpp
- meshopt_clusterlod.h
- nvhiz_vk.cpp
- nvhiz_vk.hpp
- renderer.cpp
- renderer.hpp
- renderer_raster_clusters_lod.cpp
- renderer_raytrace_clusters_lod.cpp
- resources.cpp
- resources.hpp
- scene.cpp
- scene.hpp
- scene_cache.cpp
- scene_cluster_compression.cpp
- scene_cluster_lod.cpp
- scene_gltf.cpp
- scene_preloaded.cpp
- scene_preloaded.hpp
- scene_streaming.cpp
- scene_streaming.hpp
- scene_streaming_utils.cpp
- scene_streaming_utils.hpp
- scene_textures.cpp
- scene_textures.hpp
- serialization.hpp
- uploader_interface.hpp
- - CMakeLists.txt
  - LICENSE
  - README.md
.clang-format
.editorconfig
.gitignore
CHANGELOG.md
CMakeLists.txt
CONTRIBUTING
LICENSE
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# vk_lod_clusters

This sample is demonstrating the **NVIDIA RTX Mega Geometry** technology with a continuous level of detail (LoD) technique using mesh clusters that 
leverages [`VK_NV_cluster_acceleration_structure`](https://registry.khronos.org/vulkan/specs/latest/man/html/VK_NV_cluster_acceleration_structure.html) for ray tracing. It can also rasterize the content 
using `VK_EXT/NV_mesh_shader`. Furthermore, the sample implements an on-demand streaming system from RAM to VRAM for the geometry.

In rasterization continuous LoD techniques can help performance as they reduce the impact of sub-pixel triangles.
For both ray tracing and rasterization these techniques can be combined with streaming the geometry data at
the required detail level and work within a memory budget.

This work was inspired by the Nanite rendering system for [Unreal Engine](https://www.unrealengine.com/) by Epic Games.
We highly recommend having a look at [A Deep Dive into Nanite Virtualized Geometry, Karis et al. 2021](https://www.advances.realtimerendering.com/s2021/Karis_Nanite_SIGGRAPH_Advances_2021_final.pdf).

Please have a look at the [vk_animated_clusters](https://github.com/nvpro-samples/vk_animated_clusters) to familiarize yourself with the new ray tracing cluster extension.
There are some similarities in the organization of this sample with the [vk_tessellated_clusters](https://github.com/nvpro-samples/vk_tessellated_clusters) sample.

The sample makes use of the [meshoptimizer](https://github.com/zeux/meshoptimizer) library to process
the model and generate the required cluster and LoD data. The LoD system is organized in groups of clusters whose meshes were simplified together.

![image showing continuous LoD clusters](https://raw.githubusercontent.com/nvpro-samples/vk_lod_clusters/main/docs/continuous_lod_clusters.png)

![screenshot showing a highly detailed classic architectural building](https://raw.githubusercontent.com/nvpro-samples/vk_lod_clusters/main/docs/zorah_scene.jpg)

## Continuous level of detail using clusters

For some basic description what data structures the continuous LoD system uses and how it works please look [here](docs/lod_generation.md).

In principle the rendering loop is similar for rasterization and ray tracing.
The traversal of the LoD hierarchy and the interaction with the streaming system are the same.

One key difference is that for ray tracing the cluster level acceleration structures (CLAS) need to be built,
as well as the bottom level acceleration structure (BLAS) that reflects which clusters are used in an instance.
Rasterization can render directly from the original geometry data and can render from the global list of clusters
of any instance.

To allow easing into the topic, the sample has options to disable streaming as well as simplifying the CLAS
allocation or switch between rasterization and ray tracing. In the next sections we will go over how the sample 
is organized and the key operations, what functions and files to look at.

Data structures that are shared between host and device are within the `shaderio` namespace:
* [shaders/shaderio.h](/shaders/shaderio.h): Frame setup like camera and readback structure for debugging, some statistics.
* [shaders/shaderio_scene.h](/shaders/shaderio_scene.h): Key definitions to represent the scene and cluster geometry.

The scene can be rendered with or without streaming:
* [scene_preloaded.cpp](/src/scene_preloaded.cpp): simply uploads all geometry, with all clusters of all LoD levels.
* [scene_streaming.cpp](/src/scene_streaming.cpp): implements the streaming system, more details later. Enabled by default.

The full logic of the renderers is implemented in:
* [renderer_raster_clusters_lod.cpp](/src/renderer_raster_clusters_lod.cpp): Rasterization using `VK_NV_mesh_shader` or `VK_EXT_mesh_shader`
* [renderer_raytrace_clusters_lod.cpp](/src/renderer_raytrace_clusters_lod.cpp): Ray tracing using `VK_NV_cluster_acceleration_structure`. Enabled by default if available.

The sample also showcases a ray tracing specific optimization for [BLAS Sharing](docs/blas_sharing.md).

### Model processing

This sample uses a custom fork of [meshoptimizer's](https://github.com/zeux/meshoptimizer) single header [clusterlod.h](src/meshopt_clusterlod.h) to support "inner" parallelism.

Inside [scene_cluster_lod.cpp](/src/scene_cluster_lod.cpp) the `Scene:buildGeometryLod(...)` function covers the usage of the libraries and what data we need to extract from them. The `Scene::storeGroup(...)` function takes the resulting cluster group and packs it into a binary blob used for the runtime representation. The geometry data is later saved into the cache file for faster loading and streaming.

The cluster group can be compressed using a lossless compression scheme. However, we recommend the dropping of mantissa bits for both vertex positions and UV coordinates.
The compression is done in the `Scene::compressGroup(...)` function inside [scene_cluster_compression.cpp](/src/scene_cluster_compression.cpp).

In the UI you can influence the size of clusters and the LoD grouping of them in _"Clusters & LoDs generation"_.

> [!WARNING]
> The processing of larger scenes can take a while, even on CPUs with many cores. Therefore the application will save
> an uncompressed cache file of the results automatically. This file is a simple memory mappable binary file that can take a lot of space
> and is placed next to the original file with a `.nvsngeo` file ending. During processing existing cache files will be overwritten without warning.
> If disk space is a concern use `--autosavecache 0` to avoid the automatic storage, however it is not recommended.
>
> With the `--processingonly 1` command-line option one can reduce peak memory consumption during processing of scenes with many geometries.
> In this mode saving to the cache file is interleaved with the processing and resources are deallocated immediately once saved.
> At the end of the processing the app closes automatically.
> In combination with the `--processingpartial 1` command-line option, the processing only mode can resume partial results. So one can terminate the app during processing and continue at a later time.
> There is no consistency checking of settings or input meshes for this.
> To reduce system memory usage during processing one may lower the percentage of system threads that are used via `--processingthreadpct <float 0.0 - 1.0>` (default is 0.5, half the systems supported concurrency).
>
> If system memory usage after loading a cached file is a concern, then `--mappedcache 1` can be used to load data through memory mapping directly (forced for caches that are >= 2 GiB). However, we still have to improve the streaming logic a bit to avoid IO related hitches.
>
> Be aware, there are currently only few compatibility checks for these cache files, therefore we recommend deleting if changes were made to the original input mesh.

The model loader can make use of these glTF 2.0 Extensions:
- [EXT_meshopt_compression](https://github.com/KhronosGroup/glTF/blob/main/extensions/2.0/Vendor/EXT_meshopt_compression/README.md)
- [EXT_mesh_gpu_instancing](https://github.com/KhronosGroup/glTF/blob/main/extensions/2.0/Vendor/EXT_mesh_gpu_instancing/README.md) (With the restriction that buffers referenced by its accessors must **not** be compressed)

### Runtime Rendering Operations

![image illustrating the rendering operations](https://raw.githubusercontent.com/nvpro-samples/vk_lod_clusters/main/docs/lod_rendering.png)

The key operation for rendering is to traverse the LoD hierarchy and build the list of
renderable clusters. For ray tracing we need to build BLAS based on that list as well.
When streaming is active, then CLAS have to be built for the clusters of the newly loaded groups (dashed outlines).
They are built into scratch space first, so the allocation logic can use their accurate build sizes
and move them to a persistent location.

All operations are performed indirectly on the device and do not require any readbacks to host.

Use _"Traversal"_ settings within the UI to influence it.

Relevant files to traversal in their usage order:
* [shaders/shaderio_building.h](/shaders/shaderio_building.h): All data structures related to traversal are stored in `SceneBuilding`
* [shaders/traversal_init.comp.glsl](/shaders/traversal_init.comp.glsl): Seeds the LoD root nodes of instances for traversal into `SceneBuilding::traversalNodeInfos`. Implements a shortcut to directly insert the low detail cluster into `SceneBuilding::renderClusterInfos` if only the furthest LoD would be traversed (also skips BLAS building for ray tracing).
* [shaders/traversal_run.comp.glsl](/shaders/traversal_run.comp.glsl): Performs the hierarchical LoD traversal. Outputs the list of render clusters `SceneBuilding::renderClusterInfos`.
* [shaders/build_setup.comp.glsl](/shaders/build_setup.comp.glsl): Simple compute shader that is used to do basic operations in preparation of other kernels. Often clamping results to stay within limits.
* [shaders/blas_setup_insertion.comp.glsl](/shaders/blas_setup_insertion.comp.glsl): Sets up the per-BLAS range for the cluster references based on how many clusters each BLAS needs (which traversal computed as well). This also determines how many BLAS are built at all.
* [shaders/blas_clusters_insert.comp.glsl](/shaders/blas_clusters_insert.comp.glsl): Fills the per-BLAS cluster references (`SceneBuilding::blasBuildInfos`) from the render cluster list. The actual BLAS build is triggered in `RendererRayTraceClustersLod::render` (look for "BLAS Build").
* [shaders/instances_assign_blas.comp.glsl](/shaders/instances_assign_blas.comp.glsl): After BLAS building assigns the built BLAS addresses to the TLAS instance descriptors prior building the TLAS.

**Rasterization:**
Does not need the BLAS and TLAS build steps and can render directly from `SceneBuilding::renderClusterInfos`.
Frustum and occlusion culling can be done to reduce the number of rendered clusters during traversal.
* [shaders/render_raster_clusters.mesh.glsl](/shaders/render_raster_clusters.mesh.glsl): Mesh shader to render a cluster.
* [shaders/render_raster_clusters_sw.comp.glsl](/shaders/render_raster_clusters.mesh.glsl): Compute shader to rasterize a cluster. It is only used when the "Allow SW-Raster" traversal option is active.
* [shaders/render_raster.frag.glsl](/shaders/render_raster.frag.glsl)

When `USE_DLSS` is enabled at build time, rasterization supports **DLSS Super Resolution (DLSS-SR)** to upscale a lower-resolution render target (requires `--renderer 0` or the rasterizer UI option). The motion-vector render target stays bound whenever DLSS-SR is active; only _shaded_ hardware rasterization writes motion vectors (SW-raster and other non-shaded modes disable color writes on that attachment and clear it each frame).

In some conditions (visualize == visibility buffer, culling on) one can enable the usage of a basic compute-shader based rasterizer. However it hasn't been tuned yet and in typical usage scenarios is not faster than the mesh-shader because clusters tend to have larger than single pixel triangles. You can look for `USE_SW_RASTER` in the code where it does affect traversal.

**Ray Tracing:**
After the BLAS are built, also runs the TLAS build or update and then traces rays.
Frustum and occlusion culling only influence the LoD factors per-instance through a simple heuristic. Ray tracing will render more clusters even with culling than raster.

![image showcasing the mirror box effect, a reflective box is placed into a scene of animal statues](https://raw.githubusercontent.com/nvpro-samples/vk_lod_clusters/main/docs/mirror_box.jpg)

Use the "Mirror Box" effect (double right-click or M key) to investigate the impact on geometry that is outside the frustum or
otherwise occluded.

![image show difference when path tracing is enabled](https://raw.githubusercontent.com/nvpro-samples/vk_lod_clusters/main/docs/path_tracing.jpg)

There is two sets of shaders, and their usage depends on whether _Path Tracing_ is enabled. They have a slightly different configuration. The default / basic set of shaders do shading in the hit-shader, whilst path-tracing does shading in the ray generation shader. Overall the material complexity in this sample is intentionally kept low and we recommend to have a look at [vk_gltf_renderer](https://github.com/nvpro-samples/vk_gltf_renderer) for more fidelity. Path tracing uses the physical sky model, whilst the other options use a simplified version.

Basic shaders:

* [shaders/render_raytrace_clusters.rchit.glsl](/shaders/render_raytrace_clusters.rchit.glsl): Hit shader that handles shading of a hit on a cluster. There is only cluster geometry in this sample to be hit.
* [shaders/render_raytrace_clusters.rahit.glsl](/shaders/render_raytrace_clusters.rahit.glsl): Any hit shader for alpha-masked materials.
* [shaders/render_raytrace.rgen.glsl](/shaders/render_raytrace.rgen.glsl): Ray generation shader, also implements the simple mirror effect.
* [shaders/render_raytrace.rmiss.glsl](/shaders/render_raytrace.rmiss.glsl): Miss shader.

Path tracing shaders:

* [shaders/render_pathtrace_clusters.rchit.glsl](/shaders/render_pathtrace_clusters.rchit.glsl): Hit shader that provides minimal information back to the ray generation shader.
* [shaders/render_pathtrace_clusters.rahit.glsl](/shaders/render_pathtrace_clusters.rahit.glsl): Any hit shader for alpha-masked materials.
* [shaders/render_pathtrace.rgen.glsl](/shaders/render_pathtrace.rgen.glsl): Ray generation shader that handles the multi-bounce path tracing as well as the material evaluation.
* [shaders/render_pathtrace.rmiss.glsl](/shaders/render_pathtrace.rmiss.glsl) Miss shader.

The ray tracing code path can optimize the number of BLAS builds through _"BLAS Sharing"_, which allows instances to use the BLAS
from another instance.

![image illustrating the blas sharing optimizations](https://raw.githubusercontent.com/nvpro-samples/vk_lod_clusters/main/docs/blas_techniques.png)

Please have a look at the [BLAS Sharing documentation](docs/blas_sharing.md).

The occlusion culling is kept basic, testing the footprint of the bounding box against the appropriate mip-level of last frame's HiZ buffer and last frame's matrices. This can cause artifacts on faster motion.

### Streaming Operations

![image illustrating the streaming operations](https://raw.githubusercontent.com/nvpro-samples/vk_lod_clusters/main/docs/lod_streaming.png)

Please have a look at the [Streaming Operations documentation](docs/streaming.md).

![image illustrating the separated transfers for ray tracing](https://raw.githubusercontent.com/nvpro-samples/vk_lod_clusters/main/docs/lod_streaming_rt.png)

There is one distinct difference between rasterization and ray tracing. Ray tracing will store the vertex positions into a temporary space used only during CLAS building, while rasterization will keep the positions persistently along with the group. As ray tracing allows fetching the positions at hit-time through intrinsics, there is no direct need to keep the data around.

### GPU-Driven CLAS Allocation

> [!IMPORTANT] Resizable CLAS storage through sparse buffer usage
> We are leveraging a sparse buffer for the CLAS allocation, so we can grow it on demand. For this we read back the information about
> actual CLAS sizes from the GPU-side allocation manager.
> This is crucial as the host-side estimates are way too conservative (e.g. often 3-5x bigger than actual). The CLAS 
> buffer is sized in such a way that it can fit N-frames worth of worst-case sized CLAS builds.
> The streaming manager is configured for a certain number of CLAS builds per frame, and N defines the maximum frame latency
> between GPU-produced allocation data and CPU readback/decision-making.

![image illustrating the streaming operations](https://raw.githubusercontent.com/nvpro-samples/vk_lod_clusters/main/docs/lod_allocation.png)

Please have a look at the [GPU-Driven CLAS Allocation documentation](docs/clas_allocation.md)

## Problem-Solving

The sample uses a lot of technologies and has many configurations. We don't have a lot of test coverage for it. If you experience instabilities, please let us know through GitHub Issues.
You can use the commandline to change some defaults:

* `--renderer 0` starts with rasterization.
* `--supersample 0` disables the super sampling that otherwise doubles rendering resolution in each dimension. 
* `--clasallocator 0` disables the more complex gpu-driven allocator when streaming
* `--gridcopies N` set the number of model copies in the scene.
* `--gridunique 0` disables the generation of unique geometries for every model copy. Greatly reduces memory consumption by truly instancing everything. It's on for the sample bunny scene by default, but off otherwise.
* `--streaming 0` disables streaming system and uses preloaded scene (warning this can use a lot of memory, use above `--gridunique 0` to reduce)
* `--vsync 0` disable vsync. If changing vsync via UI does not work, try to use the driver's *NVIDIA Control Panel* and set `Vulkan/OpenGL present method: native`.
* `--autoloadcache 0` disables loading scenes from cache file.
* `--mappedcache 1` keeps memory mapped cache file persistently, otherwise loads cache to system memory. Useful to save RAM on very large scenes.
* `--autosavecache 0` disables saving the cache file.
* `--processingonly 1 --processingthreadpct 0.25 --processingpartial 1` when processing big scenes, the dedicated processing only mode is better and one can reduce the memory consumption by lowering the amount of threads used.
* `--forcepreprocessmegabytes 1024` if a scene's raw geometry (vertex & indices) is greater than this cutoff, use a dedicated preprocess pass. Can be quicker and allows using memory mapped cache file. Default is 2048 for 2 GiB.
* `--multimaterials 1 --attributes 7 --texturedmaterials 1` enables textured PBR materials (see [Materials](#materials) below).
* `--maxtexturemegabytes <MiB>` sets an upper VRAM budget for material textures (default **4096**; **0** = no limit). See [Materials](#materials).
* `--dlss 1` enables DLSS when built with `USE_DLSS` (Super Resolution in rasterization, denoising in ray tracing). Use `--dlssquality <0-3>` to set quality (max performance through ultra performance).
* `--camerastring "..."` sets the initial camera (copy/paste from the _Misc Settings → Camera_ widget).
* `--addcamerapath "..."` defines a camera fly-through path (repeatable, see [Camera Path](#camera-path)).
* `--loadcamerapaths "file.txt"` replaces all paths with the definitions from a text file (see [Camera Path](#camera-path)).
* `--runcamerapath <index> <framecount>` deterministically plays back a defined path for benchmarking (see [Camera Path](#camera-path)).

## Camera Path

A camera path is a keyframed fly-through defined entirely within this sample. It can be authored in the UI, is copy/paste friendly, and can be provided on the command line just like the camera string. Its main purpose is **deterministic benchmarking**: with fixed-step playback the camera visits the exact same positions every run, independent of frame rate or GPU speed.

**UI** (_Misc Settings → Camera Paths_):

* **File:** **Save** / **Load** write and read all paths to a text file stored **next to the model file**, named after it (`<model>.camerapaths.txt`, shown under the buttons). This file is **auto-loaded** whenever that scene is loaded.
* **Path:** **New** / **Delete** add or remove a path, **Copy** / **Paste** exchange the selected path with the clipboard as a string; the dropdown below selects the active path (the index used by `--runcamerapath`).
* **Key:** **New** captures the current camera as a keyframe (inserted after the selected one); **Update** / **Delete** edit the selection. The list below shows the keyframes — selecting one previews it.
* **Smooth** uses Catmull-Rom interpolation (otherwise piecewise linear), **Loop** and **Duration (s)** affect real-time playback. When **Loop** is on and the first and last keyframes coincide, the smoothing wraps around the join for a seamless loop.
* **Play** / **Stop** / **Restart** and the **t** slider drive a real-time preview.
* **Run fixed** plays the path across **Fixed frames** frames, exactly as `--runcamerapath` does.

**Command line / benchmarking:**

Paths are defined with one or more `--addcamerapath` options; the index equals the order in which they were added (the first is `0`). `--loadcamerapaths "file.txt"` instead replaces the whole set with the paths from a text file (see the format below). Paths given on the command line are **global** and take precedence: when any are present the per-scene `<scene>.camerapaths.txt` file is not auto-loaded. `--runcamerapath <index> <framecount>` then spreads the selected path across exactly `<framecount>` rendered frames (frame `f` maps to path position `f / (framecount-1)`), so it is fully deterministic.

> Note: the per-scene file is loaded only after its scene finishes loading (which is asynchronous), so a command-line `--runcamerapath` at startup cannot reference it yet. For benchmarking either provide the path on the command line with `--addcamerapath` / `--loadcamerapaths`, or place `--runcamerapath` inside the sequences of a `--sequencefile` (which run after the scene is resident).

For benchmarking, put `--runcamerapath` in each sequence and match `--sequenceframes` to the frame count. For example a sequence script (`--sequencefile bench.txt`):

```
SEQUENCE "flythrough raytrace"
--renderer 1
--runcamerapath 0 256

SEQUENCE "flythrough raster"
--renderer 0
--runcamerapath 0 256
```

with the path stored in a file `flythrough.camerapaths.txt`:

```
smooth 1 loop 0 dur 10 ;
  {0, 2, 5}, {0, 0, 0}, {0, 1, 0}, {60} ;
  {5, 2, 0}, {0, 0, 0}, {0, 1, 0}, {60} ;
  {0, 2, -5}, {0, 0, 0}, {0, 1, 0}, {60}
```

run with:

```
vk_lod_clusters --sequenceframes 256 --loadcamerapaths flythrough.camerapaths.txt --sequencefile bench.txt
```

**String format** (also valid inside `.cfg` files as a single-line, quoted value): an optional header of `smooth <0/1> loop <0/1> dur <seconds>`, followed by `;`-separated keyframes each written as `{eye}, {center}, {up}, {fov}` (the per-keyframe fov is optional). The **Copy** button produces this canonical form. In the paths **file** a path may span any number of lines and is parsed until the next path begins (each path starts with the `smooth` header keyword); anything from a `#` to the end of a line is a comment.

## Materials

The sample supports simple colored materials, alpha-masked materials, and (optionally) textured PBR shading.
However, the shading quality is kept rather basic given the focus was geometry in this sample. For higher quality
shading with a lot more features please refer to [vk_gltf_renderer](https://github.com/nvpro-samples/vk_gltf_renderer)

### Textured PBR (metallic-roughness)

Textured PBR is **disabled by default**. When enabled, glTF **PBR metallic-roughness** materials can use base color, metallic-roughness, normal, occlusion, and emissive textures. **Specular-glossiness** materials are not supported for texturing (factor colors only).

**Command line:**

```
--multimaterials 1 --attributes 7 --texturedmaterials 1
```

**UI:**

* _Scene Modifiers → Allow textured materials_
* _Scene Modifiers → Max texture MiB_ — VRAM budget for material textures (default 4096; 0 = no limit). Reloads textures when changed.
* _Cluster Settings → Other → Mesh Multi-Materials_
* _Cluster Settings → Other → Enabled Attributes_: enable **NRM**, **TAN**, and **TEX 0** (equivalent to `--attributes 7`)
* _Rendering Settings → Other → Facet shading_: disable it.

Changing _Allow textured materials_ reloads the scene, but does not require new processing.

**Restrictions:**

* **Workflow:** metallic-roughness only (no specular-glossiness texturing).
* **Vertex attributes:** meshes must provide `NORMAL`, `TANGENT`, and `TEXCOORD_0`. Tangents are required for normal mapping.
* **Texture coordinates:** only `TEXCOORD_0` is used; glTF per-texture texcoord indices are ignored.
* **File formats:** textures must be external `dds` or `ktx2` files (same as alpha-masked materials).
* **Loading:** material textures are loaded at scene init and are **not** streamed with geometry. By default a **4 GiB** VRAM budget applies (`--maxtexturemegabytes 4096`): textures start at full resolution, and if the total exceeds the budget, finer mips are dropped in round-robin order until the limit is met. Each texture always retains at least its coarsest mip. Set `--maxtexturemegabytes 0` to load all mips unconditionally.
* **Multi-material:** glTF meshes with multiple materials per mesh require `--multimaterials 1`.

Shaders are compiled with texture sampling only when the loaded scene actually contains textured materials (`HAS_TEXTURED_MATERIALS`).

### Alpha-masked materials

Alpha-masked materials work independently of textured PBR and remain enabled by default when the glTF uses `alphaMode = MASK`. Their textures are also loaded at scene init (not streamed) and count toward the texture VRAM budget.

## Limitations

* The `ClusterID` can only be accessed in shaders using  `gl_ClusterIDNV` after enabling `VkRayTracingPipelineClusterAccelerationStructureCreateInfoNV::allowClusterAccelerationStructure` for that pipeline.
  We use `GL_EXT_spirv_intrinsics` rather than dedicated GLSL extension support that may come at a later time.
* Few error checks are performed on out of memory situations, which can happen on higher _"render copies"_ values, or the complexity of the loaded scene
* The number of threads used in the persistent kernel is based on a crude heuristic for now and was not evaluated to be the optimal amount.
* The bounding box visualizations don't show for ray tracing when DLSS denoising is active, and they will only show clusters that are part of BLAS builds in the current frame. Prefer using rasterization to see them.
* DLSS Super Resolution (rasterization): motion-vector render target is always bound; only shaded HW raster writes it. HBAO is disabled while DLSS-SR is active.
* Material textures (PBR and alpha-mask) are loaded at scene init and are not streamed, even when geometry streaming is enabled. Use `--maxtexturemegabytes` (default 4096) to cap total texture VRAM; see [Materials](#materials).
* Alpha-masked materials: 
  - Always uses texture coordinate 0 independent of glTF material's texcoord. 
  - Does require enabling texture coordinate loading (`--attributes <bitflag containing 4>` or ui).
  - May need multi-material support for glTF meshes (`--multimaterials 1` or ui). The textures must be provided as `ktx2` or `dds`. Textures are fully loaded into VRAM at scene init (not streamed).
  - See also [Materials](#materials) for full PBR texturing restrictions.
* `doubleSided` materials:
  - Are a lot slower with `EXT_mesh_shader` than with `NV_mesh_shader` on NVIDIA hardware. Primitive culling is still exclusive to NV_mesh_shader, given there is no reasonable portable and fast way for EXT_mesh_shader.
  - Are only accurately done for multi-material meshes if alpha-masking is properly enabled.
* Tangent space must be provided with the glTF meshes if normal maps are used, there is no automatic generation of these.

## Future Improvements

* Improve streaming performance
* Reduce memory consumption
* Add texture streaming for material textures (long term).

## Building and Running

Requires at least Vulkan SDK 1.4.309.0

The `VK_NV_cluster_acceleration_structure` extension is available since driver version `572.16` from 1/30/2025.
The sample should run on older drivers with just rasterization available.

Point cmake to the `vk_lod_clusters` directory and for example set the output directory to `/build`.
We recommend starting with a `Release` build, as the `Debug` build has a lot more UI elements.

The cmake setup will download the `Stanford Bunny` glTF 2.0 model that serves as default scene.

If `USE_DLSS` is activated in the cmake options, then the DLSS/NGX runtime is downloaded by cmake setup as well:

* **Ray tracing:** DLSS Ray Reconstruction (DLSS-RR) denoising for the path-traced output (`DLSS - RR` in the UI, or `--dlss 1` with the ray tracer).
* **Rasterization:** DLSS Super Resolution (DLSS-SR) upscales a jittered lower-resolution render (`DLSS - SR` in the UI, or `--dlss 1` with `--renderer 0`). Motion vectors are written only in shaded HW-raster mode.

Use `--dlssquality` or the UI quality preset to control the performance/quality trade-off.

It will also look for [`nvpro_core2`](https://github.com/nvpro-samples/nvpro_core2) either as subdirectory of the current project directory, or up to two levels above. If it is not found, it will automatically download the git repo into .

> [!IMPORTANT]
> Note, that the repository of `nvpro_core2` needs to be updated manually, when the sample is updated manually, as version mismatches could occur over time. Either run the appropriate git commands or delete `/build/_deps/nvpro_core2`.

### Interactions

* `WASD` and `QE` to control the camera. `SHIFT` for faster speed, `CTRL` for slower. `ALT` to orbit around the last hit point.
* `SPACE` or `double left click` to get a surface hit point that the camera will orient itself to. This also adjusts the walking speed based on a percentage of the distance to the point. Meaning close by hit points will cause slower walk than points further in the distance.
* `M` or `double right click` to on a surface hit point to generate the reflective mirror box in ray tracing. Trigger this on the sky to make it disappear.
* `R` to reload the shaders (meant for debugging).

## Further Samples about NVIDIA RTX Mega Geometry

Other Vulkan samples using the new extensions are:
- https://github.com/nvpro-samples/vk_animated_clusters - showcases basic usage of new ray tracing cluster extension.
- https://github.com/nvpro-samples/vk_lod_clusters - provides a sample implementation of a basic cluster-LoD based rendering and streaming system.
- https://github.com/nvpro-samples/vk_partitioned_tlas - New extension to manage incremental TLAS updates.

We also recommend having a look at [RTX Mega Geometry](https://github.com/NVIDIA-RTX/RTXMG), which demonstrates tessellation of subdivision surfaces in DirectX 12.

## Additional Scenes

### Zorah Demo Scene

This is a glTF export of the highly detailed raw geometry from the [NVIDIA RTX Kit - Zorah Sample](https://developer.nvidia.com/rtx-kit?sortBy=developer_learning_library) as [presented at GDC 2025](https://developer.nvidia.com/blog/nvidia-rtx-advances-with-neural-rendering-and-digital-human-technologies-at-gdc-2025/).

![screenshot showing a highly detailed classical building with intricate ornaments](https://raw.githubusercontent.com/nvpro-samples/vk_lod_clusters/main/docs/zorah_scene.jpg)

- [zorah_main_public.v2.gltf.7z](https://developer.download.nvidia.com/ProGraphics/nvpro-samples/zorah_main_public.v2.gltf.7z)
  - 1.63 G Triangles, with instancing 18.9 G Triangles
  - Cannot be pre-loaded must be streamed
  - **7.22 GB 7z** - 2026/3/10, unpacks to **9.32 GB on disk**
  - The render cache file will require **26 GB on disk** next to the gltf file, it will be generated on first opening of the scene.
  - If you want to process it separately in the background use the following command-line:
    - `vk_lod_cluster.exe "zorah_main_public.v2.gltf" --clusterconfig 4 --processingonly 1 --processingthreadpct 0.5 --processingpartial 1 --compressed 1`
    - This will use 50% of the local PC's supported concurrency to process the model and allow to abort and resume the processing. On a 16-core Ryzen 9 a value of `0.5` will yield 16 threads, requires 29 GB RAM (brief peak of 44 GB) and takes around 6-10 minutes. We recommend lower thread percentages on machines with less RAM.
  - **NOTE:** Older versions of this file were larger, this sample has changed the file format of its file cache. When loading an old version, the processing will be triggered automatically and the old cache file is overwritten. It can take a bit until the new file versions have been propagated to servers worldwide.

Make sure to use an NVME or SSD drive for storing these files. We recommend GPUs with at least 8 GB VRAM.

> [!IMPORTANT]
> Open or Drag & Drop the `zorah_main_public.v2.cfg` or `zorah_main_public.v2.no_mountains.cfg` file within the vk_lod_clusters application and _NOT_ the `.glTF` directly.

Known Issues:
* Compared to the original demo some of the vegetation had to be removed to make the sharing of the glTF possible (the asset itself is licensed under MIT License).
* Some objects float a bit strangely in the air and lack animation, this is expected for this scene and sample.
* The vegetation will appear to fade out a bit quickly, especially the grass. This is a known limitation for mesh-based simplifcation on
  sparse geometry like this. We do not use any techniques that preserve volume during decimation.
* Trees can appear a bit blurry with DLSS and very noisy without it. We will try to improve future versions of DLSS denoising this scenario.
* The ray tracing performance does suffer from the background mountains overlapping with the primary buildings. Use `zorah_main_public.v2.no_mountains.cfg`.

### Threedscans Statues

![screenshot showing two separate renderings of statues for humans or animals arranged on a grid](https://raw.githubusercontent.com/nvpro-samples/vk_lod_clusters/main/docs/otherscenes.jpg)

These scenes are based on models from [https://threedscans.com/](https://threedscans.com/):
- [threedscans_animals](https://developer.download.nvidia.com/ProGraphics/nvpro-samples/threedscans_animals.zip)
  - 7.9 M Triangles
  - ~ 1.4 GB preloaded memory
  - 128 MB zip 2025/7/11 (original was 290 MB zip, slow to load)
- [threedscans_statues](https://developer.download.nvidia.com/ProGraphics/nvpro-samples/threedscans_statues.zip)
  - 6.9 M Triangles
  - ~ 1.3 GB preloaded memory
  - 116 MB zip 2025/7/11 (original was 280 MB zip, slow to load)

On a "AMD Ryzen 9 7950X 16-Core Processor" processing time for `threedscans_animals` took around 10 seconds (5 unique geometries). That scene has few geometries and many triangles per geometry. Due to the few geometries the heuristic chose "inner" parallelism within operations for a single geometry at a time. Scenes with many objects will typically use "outer" parallelism over the unique geometries and tend to be processed faster overall.
By default the application now stores a cache file of the last processing (`--autosavecache 1`).

## Third Party

[meshoptimizer](https://github.com/zeux/meshoptimizer) is used for many operations, such as building the cluster lod data structures along with the mesh simplification and re-ordering triangles within clusters.

[vulkan_radix_sort](https://github.com/jaesung-cs/vulkan_radix_sort) is used when "Instance Sorting" is activated prior traversal.

vk_lod_clusters

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

vk_lod_clusters

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README