Kinemation-Skinning-Prototype

KO | EN

gitlite — search

#javascript #python #ai #typescript #llm #go #android #linux #nodejs #macos #awesome-list #pytorch #ios #open-source

Kinemation-Skinning-Prototype

(★ 36)

Unity DOTS custom skinned mesh rendering prototype for the Latios Framework

Kinemation-Skinning-Prototype 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

## Kinemation Skinning Prototype

This is a Unity DOTS project built upon
[Puppet-Test](https://github.com/keijiro/PuppetTest) for the purpose of
developing a new skinning solution known as Kinemation as part of the Latios
Framework.

This prototype is **NOT** ready for production.

The procedural animation is still done with 10 reference game objects, but these
are used to drive 10k entity dancers.

The latest master commit uses DOTS 0.50 and Unity 2020.3.30f1.

## How to Test

The project contains two important scenes: *Party DOTS* and *Part DOTS
Optimized*. The former uses an entity per bone in each dancer. The latter uses
dynamic buffers to represent the bones.

Each scene contains a *Toggle* game object with two checkboxes. If *Enable
Hybrid* is checked, skinning and rendering is handled completely by the Hybrid
Renderer using compute deformations. If it is unchecked, skinning and rendering
is handled by the Kinemation solution, which uses a different compute
deformation design. If *Enable Blending* is checked, each dancer entity will
blend randomly between two of the ten reference game object-based procedural
dancers. This can be quite expensive. If unchecked, each entity dancer will copy
the pose of one of the reference procedural dancers.

### Important

Do not modify the toggle checkboxes while in play mode. The controls affect
conversion.

## Design and Goals

Kinemation was designed primarily with two goals in mind:

-   Provide an API and backend systems for driving skeletal animation from
    normal ECS systems (no animation graph required).
-   Improve GPU performance by sacrificing some CPU performance.

In Kinemation, skinning is driven by skeletons, and there are two types:

-   Exposed – Each bone is an Entity
    -   Intuitive for programmers and saves development time
    -   Much more viable with DOTS than with game objects
    -   Automatic hierarchy updates via Transform System
    -   Great for procedurally-generated data
-   Optimized – Bones are stored in packed Dynamic Buffers
    -   Faster on CPU than Exposed
    -   Better data locality when animation logic is simple
    -   Hierarchy updates dispatched manually
    -   Exported bone support
    -   Great for fully authored data

### Differences from Unity’s Animation and Hybrid Renderer Packages

For optimized hierarchies (what the animation package always uses), Unity stores
the bind poses and hierarchy information in dynamic buffers, while Kinemation
stores this information in blob assets.

Unity requires meshes be marked as Read-Write enabled at runtime. Kinemation
only requires read-write access at conversion time, as it stores skinning data
in a blob asset using a custom compute-optimized data structure.

Unity handles bone hierarchies in root space, and reparents all skinned meshes
to the root. This matches the Skinned Mesh Renderer behavior of game objects.
Kinemation operates in skeletal reference space and leaves the hierarchy as is.

Unity uses `Immutable` compute buffers and creates a compute shader dispatch for
each unique mesh. Kinemation uses `SubUpdate` compute buffers for per-frame data
and a single compute shader dispatch can process many different meshes at once.

### Unity Exclusive Features

Unity’s solution currently supports Blend Shapes. These are not currently
supported in Kinemation but will be in a future release. Similarly, Unity
supports 4-bone vertex skinning. Kinemation is compute-only.

Unity has occlusion-culling. Kinemation disables this for the entire project.
More research is required to make it work correctly with Kinemation.

Lastly, Unity has fully animation clip sampling and blending system with
authoring tools and workflows. This prototype version does not have anything
like that. The first release version will have simple code-driven animation clip
support. Unlike Unity, runtime conversion will be supported (assuming that’s
still a thing by then).

### Kinemation Exclusive Features

Kinemation has true frustum culling support for skeletons. This means that
animated meshes will have proper bounds updated every frame. In addition, only
meshes that pass culling and LOD selection will be skinned on the GPU. Note that
frustum culling is per-skeleton rather than per-mesh.

Kinemation has a rebinding system which allows for fast and intuitive binding
and rebinding of meshes to skeletons at runtime as long as the mesh was designed
for the same number of bones as the skeleton. This is useful for things like
character customizations where all customizations are skinned to the same base
skeleton in the DCC package.

Kinemation does not require any bones to be exposed during conversion of an
optimized skeleton.

## How It Works

Kinemation has five primary entity archetypes:

-   Skinned RenderMesh
-   Exposed Bone
-   Exposed Skeleton
-   Exported Bone
-   Optimized Skeleton

The data associated with these archetypes are designed to minimize random memory
accesses where possible.

It is also worth noting that Kinemation makes use of several features of the
Latios Framework. Also, the current systems are not all placed in the most
correct locations in the player loop for simplicity of prototyping. However, now
that the design is mostly complete, this section will typically call out the
intended correct locations for these systems.

### Conversion

Currently, a single conversion system is responsible for generating all
components and blobs. Each Skinned Mesh Renderer gets classified as belonging to
an exposed skeleton, an optimized skeleton, or an invalid skeleton. Conversion
then typically involves converting the entire skeletons and all skinned meshes
using it at once. The shared meshes get converted separately to blobs.

For optimized skeletons, the bone hierarchy data needs to be extracted. A
temporary Game Object gets instantiated and
`AnimatorUtility.DeoptimizeGameObjectHierarchy()` exposes all the underlying
transforms as `GameObjects` for analysis. Currently, a hack component exists for
identifying exposed bones, and this solution likely won’t work for prefabs
referenced in subscenes.

### Binding Reactive System

`SkeletonMeshBindingReactiveSystem` is the first system to process converted
entities at runtime. As its name suggests, this is a reactive system. It
generates a sync point when necessary and reacts to new and deleted skeletons as
well as new and deleted meshes. It also reacts to rebindings via changes to the
`BindSkeletonRoot` component, although this does not incur a sync point.
Existence and bindings are assumed to be relatively persistent, so this system
tries to perform many expensive random-access operations in order to avoid such
operations in per-frame updates.

One of these operations is skinning source data allocation inside the two mesh
compute buffers. One compute buffer stores the mesh vertices, while the other
stores the weights. New and dead mesh blobs get collected into an
`UnsafeParallelBlockList` and an `IJob` processes this during the sync point.
This job maintains a `MeshGpuManager` collection component which handles all the
occupied and free ranges of meshes as well as reference counts. It also
generates an upload command list for a future system.

The second operation is exposed skeleton culling index linking. Each unique
exposed skeleton gets assigned an index which needs to be synchronized with all
of its exposed bones. These indices will be used to index bit arrays for
optimally passing culling information from bones to the skeleton, avoiding
expensive cache-thrashing random accesses.

The third operation is the actual binding changes between meshes and skeletons.
These are also recorded in an `UnsafeParallelBlockList`. The recorded operations
are sorted and batched during the sync point. Then after the sync point, a job
runs through all skeletons and updates all data related to the bindings.
Skeletons keep a list of bound mesh entities in the dynamic buffers. They also
cache references to GPU memory for the mesh skinning source data. And lastly,
each mesh blob contains bounding spheres for the mesh for each bone. The
skeletons use this to bake bounding spheres for all bound meshes for each bone.
In the case of exposed skeletons, this also gets forwarded to all the individual
bones via random access.

This system is set to run in the presentation structural changes system group.
However, it can be run multiple times per frame at different locations if moving
its sync point is beneficial.

### Optimized Skeleton Bone Export

Optimized skeletons have the ability to export bone transforms to external
entities. Currently, this export is done using a `LocalToWorld` `[WriteGroup]`
override. However, this will be replaced to use `LocalToParent` for the official
release. The change will remove the need for a system like
`TempFixExportedTransformsSystem` to be scheduled so precariously around the
`TransformSystemGroup` when exported bones have children and skeletons have
parents.

### Skeleton and Bone Bounds Update

`SkeletonBoundsUpdateSystem` is responsible for updating both optimized skeleton
and exposed bone bounds. Optimized skeletons calculate a world-space AABB for
the entire skeleton hierarchy. Exposed bones calculate a world-space AABB for
each bone individually. Both archetypes update a chunk bounds component when
necessary. Change and order version filtering are both used.

### Compute Deform Buffer Allocation

Kinemation makes use of the same Shader variables and skinning buffer format as
Unity. That means that any shader graph shader using the Compute Deformation
node or any custom shader compatible with Hybrid Renderer’s compute deformation
skinning will be compatible with Kinemation. Each `RenderMesh` gets a material
instance property with an index into a compute deformation vertices buffer. This
buffer reserves memory for all deformed vertices for all deforming `RenderMesh`
instances in the ECS World. A downside to this is that this buffer gets huge and
wasteful. More research is needed to resolve this.

Because the buffer memory is valuable, indices are reassigned every frame to
reduce fragmentation. `ChunkComputeDeformMemoryMetadata` maintains state for
this process. `UpdateChunkComputeDeformMetadataSystem` looks for changed meshes
or structural changes and caches vertex counts and chunk counts in the chunk
component. These results are used by `AllocateDeformedMeshesSystem` which
iterates the meta-chunks in a single-threaded job and computes prefix sums at
the chunk level. Iterating over meta-chunks like this is incredibly fast and
effectively removes the performance penalty of a single-threaded prefix sum.
Once the prefix sums are computed, a parallel job distributes unique vertices to
the individual entities.

It is worth noting the buffer index assignment assigns lower indices to meshes
rendered the previous frame. This was an experiment and currently serves no
practical purpose.

### Per Frame Setup

The Hybrid Renderer uses the `BatchRendererGroup` API to provide drawing
information into the engine. This API invokes a culling callback each time
culling needs to occur. Culling occurs multiple times per frame, usually while
the Scriptable Render Pipeline code is executing. When profiling play mode in
the editor, three callbacks happen per frame. One is for the main camera, one is
for shadows, and one is for the scene view.

Kinemation dispatches skinning during these culling callbacks in order to avoid
skinning invisible meshes. Skinning only has to happen once per visible mesh
across all culling callbacks, so Kinemation caches skinning state in components
on both skeleton and mesh entities. This data needs to be reset each frame prior
to culling callbacks. `ClearPerFrameSkinningDataSystem` performs this.

Skeletons cache which bone matrix buffer and bone offsets they used when first
skinning a batch of meshes. This is useful when different mesh LODs bound to the
same skeleton are selected during different culling callbacks.

Meshes store three Booleans in a byte-sized component. One of these is the
selected LOD. This never gets cleared and is kept up-to-date in the LOD
selection system. The second is whether the mesh should be rendered for a given
callback. This is cleared for every callback and also at the beginning of the
frame. The third flag specifies whether the mesh was rendered at least once
during the frame by a previous callback. When this flag is set, the mesh has had
a skinning dispatch for the frame.

### Source Mesh Data Upload

All Compute Buffers are pooled using a type called `ComputeBufferTrackingPool`.
A Collection Component wraps this and exists on the `worldBlackboardEntity`.
`BeginPerFrameMeshSkinningBuffersUploadSystem` is responsible for creating this.
It updates the pool using async readback fences to ensure `SubUpdate` buffers
get recycled properly. It then reserves mesh buffer sizes and populates the
source mesh upload buffers in jobs. However, it does not submit the buffers, but
rather leaves that state in a `MeshGpuUploadBuffers` collection component for
the next user to finish and dispatch. Usually, the buffers writes are committed
and the upload compute shaders are dispatched during the first culling callback.
However, if no culling callback is performed, this system will perform these
steps at the beginning of the next frame to ensure the GPU stays in sync with
the CPU allocation management.

### Latios Hybrid Renderer Replacement

Unfortunately, Unity’s Hybrid Renderer does not support custom culling. This is
something they are working on, but `Kinemation` couldn’t wait. The solution is
to replace Unity’s Hybrid Renderer with a custom one using a different culling
callback.

This replacement does several things differently. First, it forces all batch
bounds to the max extents (1e9f) for any chunks containing skinned meshes.

Second, it creates the `KinemationCullingSuperSystem`.

And third, in the culling callback, it places all the callback arguments into a
collection component and then invokes `Update()` on the
`KinemationCullingSuperSystem`. Once the update finishes, it returns the
collection component’s `JobHandle`.

### Skeleton LODs Update

Inside the culling callback, the first system to run is
`UpdateSkinnedLODsSystem`. This system runs what is essentially the LOD update
job from Unity’s Hybrid Renderer. However, there are a few modifications.

First, the LODs have to be propagated to the per-mesh-entity flags so skeleton
culling can easily look them up via entity in a job.

Second, Unity’s implementation had some issues managing state and change filters
using `EntityQuery` dependencies. But with a proper system, this could be
corrected. Admittedly, these fixes haven’t really been tested yet. But the
opportunity is real.

### Per Camera Setup

The second system to run is `ClearPerCameraFlagsSystem`. And by this point, it
should be pretty obvious what this system does. It clears the per-camera flag on
the byte-sized component attached to skinned meshes.

What is less obvious is that it also assigns its `GlobalSystemVersion` to an ICD
on the `worldBlackboardEntity`. This allows future systems to detect changes to
the flags relative to this system.

### Frustum Culling and Skinning

`SkeletonFrustumCullingAndSkinningDispatchSystem` should probably be split into
multiple systems. But for now, it exists as a mega system and does many things.

First, it performs culling on exposed bones. Bones which are visible mark a bit
in a per-thread `UnsafeBitArray` associated with their exposed skeleton culling
index, which was set up by the binding reactive system. For 10k skeletons, each
thread has 1.3 kB of bits to potentially mark. This easily fits in L1 cache
making these random-access marks extremely cheap. And that’s important, because
these marks have to happen for each bone. For a skeleton that is visible, nearly
all bones in it will have to mark their visibility.

After this job, the `UnsafeBitArrays` get collapsed into one using bitwise-OR
operations.

The third job is `CullAndCollectMeshMetadataJob`. This job identifies visible
skeletons and queues up skinning metadata commands. For exposed skeletons, it
uses the `UnsafeBitArray` to determine visibility. For optimized skeletons, it
performs frustum culling directly on the optimized skeleton bounds. For each
exposed skeleton, it looks up all mesh flags and checks their state. Meshes yet
to be skinned have the skeleton index in chunk and the mesh dynamic buffer index
stored in a `NativeStream` along with the mesh’s Compute Deform material
property index. For all visible meshes with enabled LODs, flags are updated to
mark the mesh as visible per frame and per camera. The job also stores counts
relative to the number of meshes requiring skinning as well as which bone matrix
buffer to use.

The fourth and fifth jobs prefix sum the counts so that a metadata compute
buffer can be populated with the correct offsets in parallel.

The sixth job is `WriteBuffersJob`, and it calculates the skin matrices and
writes them to a compute buffer. It also writes the metadata buffer for
specifying offsets to the skeletons and meshes the compute shader needs to
index. All of this data has either been calculated via the prefix sums or was
cached in the skeleton. For exposed skeletons, this job pays the cost of
random-accessing the `LocalToWorld` of all the individual bones, but it only
does so for skeletons that require skinning.

While all these jobs are running, the main thread stays busy juggling collection
components, dispatching mesh uploads, and priming the compute skinning shader.
Finally, it force-completes all the jobs so that it can end the writes to the
compute buffers and dispatch the compute skinning shader. It finally binds the
deformed data to the global buffer.

### Unskinned LODs Update

Because skinning requires force-completing jobs, unskinned LODs update in a
separate system after skinning dispatch. This is the same as the skinning LOD
update, except there are no skinning-specific flags to update.

### Batch Renderer Group Visibilities

The last system to run is `CullUnskinnedRenderersAndUpdateVisibilitiesSystem`.
This system does the frustum culling logic that the Hybrid Renderer normally
does. However, it only does this check for the unskinned meshes. The Hybrid
Renderer updates the batch renderer group visibilities in the frustum culling
job, so this job does the same. However, for skinned meshes, instead of chunk
bounds and world bounds test, this job uses the change version since the flag
clearing and then checks the flags for chunks that changed.

One other aspect to this system is that it uses temp memory allocated in the job
for scratch indices rather than allocate all the memory prior to the job. This
is a little faster.

### The Batch Skinning Shader

With all the systems discussed, there’s only one piece left in the puzzle, and
that is the compute shader. This thing is weird, but fast. It takes advantage of
two aspects common to most GPUs.

First, many GPUs have dedicated on-chip memory which maps to HLSL’s
`groupshared` variables. In DX11, this memory is required to be 32 kB per thread
group. That is enough to fit 682 bone matrices. So as an optimization, each
skeleton is assigned a threadgroup. Skeletons with less than 683 bones will
store the matrices in `groupshared` memory at the beginning of the shader and
then run a group barrier. After this, the thread group processes all meshes
bound to the skeleton, using the cached matrices for fast lookups. This
optimization is especially fast on AMD and Nvidia GPUs. But modern Adreno
(Qualcomm) GPUs, PowerVR GPUs (older iOS devices), and Apple designed GPUs
(modern iOS and Apple Silicon) all have this hardware. Even very recent intel
integrated graphics adopted this on-chip memory in their GEN11 architecture.
That just leaves ARM Mali and older Intel integrated GPUs where this
optimization doesn’t work. But that can be detected at runtime so Kinemation can
use a slightly different shader for those cases. This isn’t implemented yet in
the prototype. One other caveat is that the threadgroup size is set to 1024
threads, which is optimal for desktop GPUs but might be too high for mobile.
Fortunately, the shader can be easily tweaked to work with smaller thread
groups.

The second optimization is with how bone weights are stored. Traditionally, all
bone weights for a given vertex are stored adjacent in memory. A second buffer
specifies the starts and counts for the weights. There are two problems with
this. The first is that we need two buffers just for weights. The second is that
this thrashes GPU caches. Unlike CPU caches, the GPU caches are designed for
throughput and coalesce reads and writes across multiple GPU threads. In other
words, data locality is emphasized and temporal locality is not really a thing.
A batch of threads in a thread group work in lockstep for evaluating the first
weight, and then the second, and then the third, and so on. So naturally, the
GPU could benefit if all the first weights for a batch of threads were packed
together, and then the second, and so on. And so that is how Kinemation stores
the weights. It steals the high bits of the bone index to encode linked-list
offsets to the next weights for a given vertex. It also uses a negative weight
to signal that it is the last weight. And batches of vertices are prefixed with
batch linked list to help threads along. Much of this logic relies on a shared
threadgroup scalar processor which many GPUs also have and get further speedups
from. This is known as “Scalarization”.

My personal GPU is an R9 390, which is very similar to the GPU in the PS4 Pro.
One thing about these GCN GPU architectures is that to get good GPU utilization
with max `groupshared` memory usage, only 32 vector registers (VGPRs) can be
used. Unlike Unity’s compute shader which uses 36, this one uses between 26 and
28\. To be fair, Unity uses smaller threadgroup sizes where this doesn’t matter
as much. But still, achieving high utilization is possible, even when processing
multiple meshes in a single threadgroup.

One last thing, each bone weight has 7 unused bits that I don’t know what to do
with. Any ideas?

## Performance

Regarding performance, while I made a strong effort to keep CPU costs down as
much as possible, my goal was to improve GPU performance.

### GPU Performance

The benchmark only involves one skinned mesh per skeleton with that mesh being
shared by all entities. This is best-case-scenario for the Hybrid Renderer
skinning. Yet despite this, between 1k and 10k dancers, the simulation will
become GPU-bound and the main thread will block on compute buffer uploads. At
10k dancers, frames take between 25 and 30 milliseconds on my system.

But with Kinemation, the frustum culling significantly reduces the GPU load. 10k
dancers are CPU-bound running well-above 60 FPS. And Windows reports GPU
utilization at only 50% compared to the 100% with the Hybrid Renderer. The
Hybrid Renderer does perform frustum culling on the mesh instances, so this
difference is entirely from compute skinning.

I also tested a slightly modified simulation where frustum culling was disabled
to compare raw skinning performance. Kinemation skinning is 1.5x faster.

### CPU Performance

This is where things get tricky. When not GPU-bound, Kinemation performs worse.
But that’s not because the algorithms are slower. It is because Kinemation is
doing a lot more work per frame. It is doing bounds updates for all the bones in
the skeleton, which adds up to 220k bones. It also endures a significant
constant overhead in the culling callbacks since it runs multiple systems in
there.

One possible optimization may be to replace the exposed bone culling algorithm
to use bounding spheres instead of bounding boxes.

Another optimization would be running more main-thread logic in Burst.

Lastly, dynamic buffer capacities were left at default values. These should
probably be tuned to reflect common use cases.

In any case, if you have ideas to further improve performance, please reach out
to me!

## What’s Next?

There is quite a bit of work remaining before Kinemation is ready for release.
Here are the planned changes:

-   Basic animation clip support
-   Skinning shader variants for up to 32k bones
-   Skinning shader variants for specific GPUs
-   Full renaming pass to improve on API consistency
-   Properly ordered systems and an installation mechanism
-   Removal of the Hybrid Renderer skinning mode
-   Support for more prefab configurations
-   SubScene and optimization support
-   Exported bones using `LocalToParent` instead of `LocalToWorld`
-   Project configuration validation
-   Documentation

## Licenses

Kinemation and Latios Framework are licensed under the Unity Companion License.

Everything else including all assets in Assets/_Code and Assets/Scenes and
Prefabs is [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

Kinemation-Skinning-Prototype

Kinemation-Skinning-Prototype 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Kinemation-Skinning-Prototype

Kinemation-Skinning-Prototype 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README