Animation Retargeting#
Warning
Animation Retargeting is currently in Beta. For more information on pre-release and beta terms, please see the Omniverse License Agreement
Overview#
The Animation Retarget Extension is a set of tools to setup characters to reuse skeletal animations. Animations are authored on one character and then can be applied to other characters of different proportions, joint names, hierarchies, and more.
The extension features a suite of features to aid with setting up Retargeting, and also methods to automate what is a traditionally challenging process.
Note
Only Retargeting of the biped morphology is supported at this time.
Setup#
Enabling Retargeting#
In order to begin using this extension, ensure omni.anim.retarget.bundle is enabled in your Omniverse KIT based app.
Note
See Omniverse Extension manager documentation on how to enable extensions.
Once enable in the Extension Manager, the window can be invoked by navigating through the menu: Window > Animation > Retargeting.
Mapping Presets#
For users of popular third party character systems, like Reallusion’s Character Creator 3 or RenderPeople, mapping files are included. With this feature, characters will automatically have their specific rig system identified, and get all the right settings applied for a proper retarget. This is done via preset files that are stored within the Retargeting extension.
To utilize this feature: with the character in the scene, go to the Retargeting panel, and select your character in the drop down menu. If its a supported character system, the tagging and facing will be populated, and a blue notification will appear in the lower right of the viewport.
Even if the character is not supported for Auto Setup, the Retargeting system can still attempt to do as much of the setup as possible. This capability can be enabled/disabled in the Auto Setup are in the UI’s Auto row.
Selection#
The Selection area shows which character is currently being setup or edited for Retargeting. The drop-down menu shows all skeletons currently in the scene.
If Retarget settings were changed and the file unsaved, there will be a bracketed asterisk Skeleton[*]. Retargeting information needs to be saved along with the scene to be reused elsewhere.
The current character can also be set via the Assign Selected Skeleton arrow button.
To focus the viewport on the character, select the target button to Focus.
Tagging#
Tags tell the retargeting system how to remap sections of the body across different characters. For each tag, a joint is specified. 5 joints are required to get an initial retarget:
Joint |
Description |
|---|---|
Head |
Joint at the base of the skull |
Left Hand |
Joint in the left wrist at the base of the hand |
Right Hand |
Joint in the right wrist at the base of the hand |
Left Foot |
Joint at the left ankle |
Right Foot |
Joint at the right ankle |
From these 5 tags, the system figures out the extent of each body section by walking upward in the hierarchy until one section collides with another.
In the synoptic, circles with solid outlines are required. Circles with dotted outlines are optional. Yellow circles denote a missing tag.
Additional tags for the Body tags can be added via the button for Add Tag > Body. These additional tags are represented in the blue synoptic as circles with dotted borders. Adding more tags for different parts of the skeleton can improve the retarget quality. However, the same body tags will need to be added to all characters in your project that are either the source of the retarget, or the destination. If additional tags are missing when retargeting, they will not be utilized.
Custom Tags are also supported. Custom tags can be added through the button for Add Tag > Custom. For example, you might have a character that has a tail and you’d like to retarget that animation to another character with a tail. Each character will need a Custom tag created for the tail, use the same tag name, and have the tip of the tail as the tagged joint.
Facing Direction#
The Retarget system needs to know two key facing directions for the character: the “Up” and the “Forward” direction the character is looking.
The “Forward” direction is visualized by the color red, and the “Up” direction is visualized with the color red.
A viewport widget is provided to help visualize what the two facing directions are currently set to. They can be toggled by clicking the blue and red “eye” icons next to each menu.
Tip
The facing viewport widget can be hidden on a per-axis basis by clicking the blue and red “eye” icons in the UI.
Retarget Pose#
The Retargeting system assumes that the source and destination characters have matching poses. By default, the system uses the rest transform of the skeleton. However, not all characters are built in the same default pose. In order to get a high quality retarget, some adjustments might need to be made to the source or destination character’s retarget pose.
The Retarget Pose is a custom pose that is stored with the character’s retarget settings. There are buttons and features to aid in viewing and editing this pose:
Option |
Result |
|---|---|
View |
View the character’s currently set retargeting pose |
Reset |
Reset to rest transform (deletes the existing retarget pose) |
Apply |
Save the current pose as the retarget pose |
Match |
Match to the pose of the specified skeleton in the menu drop-down (auto pose). |
If the desired pose is not achieved with the automatic methods, joints can manually be manipulated in the viewport to get the right pose. However, this workflow can be tedious and is recommended for advanced users.
Auto#
The Auto area contains a list of the processes used to automatically setup your character for Retargeting. If checked, these processes will be run when a character without retargeting is selected in the panel’s drop-down.
Processes can be executed on-demand as well by marking the desired checkboxes and pressing the Run button.
Process |
Result |
|---|---|
Use Mapping Files |
Utilize JSON to setup characters from popular character systems |
Facing |
Attempt to detect the Facing and Up directions of the character |
Tags |
Attempt to detect and set the 5 core tags needed to retarget |
Match |
Attempt to automatically match pose when retargeting from source to destination |
Preview#
To verify if your setting has been correctly setup, the Preview feature can be utilized.
There are 2 methods for this - Selecting default animation(s) - Select another skeleton in the retarget window and enabling preview by clicking Start Preview.
The Preview feature will give you a quick visual on how your retarget is looking. An idle, wave, and walk animation are included with the Retargeting extension, and load in a default Biped character that is also setup for retargeting.
The Start Preview button will load the character in and play the specified animation, and then also attempt to play it on your currently selected character in the Retargeting panel. The Stop Preview button will unload the preview character and return your Retargeted character to the default pose.
Other characters from within the scene can also be used as preview sources. This requires both source and destination have the same tags, similar retarget pose, and correct facing to be set up.
Warning
Using the Preview may slow framerate. Also, using skeleton preview with Retargeting will not represent any post-retargeting modifiers set on the skeleton – it will only provide a snapshot of the Retarget.
Retargeting with SkelAnimation#
The most straightforward way to use Retargeting is the SkelAnimation method within USD. SkelAnimations can by assigned by selecting the Skeleton prim under a SkelMesh, and then assigning a clip within the scene or via reference in the Property panel.
To do this, we have to specify to the SkelAnimation system what exact character the animation was created for initially. This is important, as the relationship between the initial character and animation is used to calculate the offsets for our final destination character. This link is is called Skeletal Binding.
Ensure your SkelAdaptor is enabled by loading Hydra.UI extension, and enabling Hydra SkelAdaptor as in the picture.
To setup Skeletal Binding, you’ll need an animation and the character it was made for in your scene. Next, right click on your animation in the Stage and go to Add > Animation > Skeletal Binding. In the Property pane, use the Add Target to find the character’s skeleton in the scene. The animation is now ready to Retarget on to other characters, and can be directly applied to them via the Skeletal Animation property on the Retarget character’s skeleton prim.
Multiple animations can be setup for Skeletal Binding at once by multi-selecting and adding the Skeletal Binding and setting the property.
Tip
To maintain the Binding across different use cases, bring all animations into one scene along with the default character, and then reference that into other scenes. This way, the Skeletal Binding only has to be applied once. The source character can be hidden if needed.
Retargeting with Animation Graph#
Retargeting can also be processed using the Animation Graph runtime animation system.
Note
Retargeting will execute after the Anim Graph final pose has processed.
When setting up an Animation Graph prim, a skeleton must be specified. Retargeting needs to be setup on this character for the changes to carry through the Retarget system and on to other characters that are also setup for Retargeting.
The Retargeter works automatically if the Animation Graph is using a different skeleton than the graph was authored for. For example, if you have an Animation Graph with Skeleton A, but if you apply that Animation Graph prim to Skeleton B, it will automatically apply Retargeter.
Once both characters are setup for Retargeting, the Animation Graph simply needs to be applied to the destination character via right clicking on the SkelRoot in the stage view and clicking Add > Animation > Animation Graph. For more information, please review the documentation on the Animation Graph
Tips & Limitations#
It is strongly recommended all of your characters (including animation source characters) are setup with a root joint. This joint is directly under the Hips or Pelvis, and on the ground plane.
Animating translation/scale won’t retarget properly, Scale is always 1 (Actual Size).
“T-Pose” – where the character’s legs are straight down and arms and fingers straight out to the side – is highly recommended as the base pose for all your character authoring. Doing so will make Retargeting’s Auto Setup faster.
It’s recommended that both source and destination characters included in your Retargeting pipeline should all face the same direction. It’s recommended that all character skeletons face in the direction of positive Z.
Sometimes it’s easier to set the source character pose to match the destination character pose. This will benefit by not having to set numerous destination character poses. For example, if the source character and animations all face -Z, and the destination characters downloaded online all face +Z, then it would be easier to adjust the source character facing to match, rather than adjusting many of the destination character directions.
Samples#
Sample scenes using Retargeting with both SkelAnim clips and the Animation Graph are available in the Animation Graph Samples panel. They can be accessed via menu: Window > Animation > Animation Graph Samples. The Anim Graph extension (omni.anim.graph.bundle) must be loaded to access this panel.
