Omni.Anim.People

Warning

Omni.Anim.People is in Beta. For more information on pre-release and beta terms, refer to the Omniverse License Agreement.

Omni.Anim.People is an Extension for simulating human events in environments such as retail stores, warehouses, and traffic intersections.

A simulation consists of a sequence of actions for each character in the scene to perform. There are five supported character actions:

• Idle

• Look around

• Queue

• Sit/Stand

• Walk

Enable omni.anim.people

1. Open the extension manager - Window > Extensions. In the extension manager, search for “people” and enable the omni.anim.people extension.

2. Load the People Simulation UI by navigating to Window > People Simulation.

Note

Click the auto-load button in the extension manger to have the extension auto-loaded on startup.

Getting Started

Basic Usage

1. In the People Simulation UI, copy and paste the following text in the Command Text Box.

   Spawn Tom
   Spawn Jerry 10 0 0 0
   Tom GoTo 10 0 0 _
   Jerry GoTo 0 0 0 _
   

2. Click the Load Characters button to load the characters assets and animations required for the command.

3. Click on Setup Characters button to attach Behavior Scripts and Animation Graph to the characters.

4. Turn off the Navmesh Based Navigation setting and click Play to run the simulation.

Note

Navmesh Based Navigation needs to be turned on for static obstacle avoidance. However, it requires a NavMesh to be built for the stage. Here, it gets off just to show how you can get the simulation running.

Run A Demo Simulation

The following are a few sample simulations to help you get started.

1. Open a sample by navigating to [Isaac Sim Assets Path]/Isaac/Samples/PeopleDemo/SimpleEventSimulation in the Content panel.

Note

[Isaac Sim Assets Path] is the path to Isaac Sim Assets. You need to set up Nucleus to use them. It can also be retrieved by printing omni.isaac.core.utils.nucleus.get_assets_root_path() with the Isaac Sim script editor.

2. Double-click simple_event_simulation.usd and select Open Original File to try out the sample.

   

3. In the People Simulation UI, click the folder icon for Command File Path.

   

4. Navigate to [Isaac Sim Assets Path]/Isaac/Samples/PeopleDemo/SimpleEventSimulation and select the commands text file.

   

5. To set up characters, click on the Setup Characters button in the People Simulation window.

6. Open the NavMesh menu and build the navmesh by clicking Bake.

   

   Note

   You can turn off the NavMesh auto-bake to increase the performance. Go to Window > Navigation > NavMesh and turn off Auto-Bake in the NavMesh settings. Otherwise, NavMesh will be baked every second by default and you can skip this step.

   

7. Click Play to run the scene.

8. Below is the expected result:

   

Command Guide

A simulation contains a sequence of commands for every character. The command structure is character_name action action_params. Below, we list each command and its input format.

Note

All characters need to be allocated under the Character Parent Prim. By default, the Character Parent Prim path is /World/Characters. Each character_name must match the name of the character’s root prim.

Spawn

Spawns a character at the given position and rotation. If the specified character_name does not match an existing character asset, a random character asset is loaded. Spawn also loads other components and assets required for executing commands such as an animation graph and character animation.

command structure:
    Spawn character_name x y z rot

example:
    Spawn Character 10 10 0 90

GoTo

GoTo moves the character to a location. GoTo can be followed by a single point or a sequence of points, the last point must specify the ending rotation that the character has when it reaches its destination. You can use ‘_’ for the ending rotation, if you prefer to not set a rotation.

command structure:
    character_name GoTo x y z end_rot

    character_name Goto x1 y1 z1 x2 y2 z2 x3 y3 z3 end_rot

example:
    Character_01 GoTo 10 10 0 90

When a sequence of points is given, the character will cross every point. If Navmesh Navigation and Dynamic Obstacle Avoidance is on, then the character tries to avoid static and dynamic obstacles.

LookAround

LookAround makes the character stand in the same spot, while moving its head from left to right. LookAround takes a duration value and performs the action for that duration.

command structure:
    character_name LookAround duration

example:
    Character_02 LookAround 10

Idle

Idle makes the character stand still. Idle takes a duration value and performs the action for that duration.

command structure:
    character_name Idle duration

example:
    Character_03 Idle 10

Sit

Sit makes the character go to a prim, and attempt to sit on it. After the specified duration ends, the character stands back up.

command structure:
    character_name Sit seat_prim duration

example:
    Character_04 Sit /World/Chair 5

Note

The Sit action plays a fixed animation, so the seat_prim must be scaled to fit the animation. Additionally, the seat_prim must contain two child prims: walk_to_offset and interact_offset. The walk_to_offset translation specifies the location where the character will walk to, and its rotation defines the character’s final orientation before interacting with the chair. The interact_offset specifies the character’s location and rotation when the sitting animation is played. For example, refer to Multi_Event_Simulation in the samples folder.

Queue

For the Queue command, you must define a queue. A queue can be defined in the command file as follows:

command structure:

    # defines the queue
    Queue Example_Queue

    # Defines each spot in the queue. Structure Queue_Spot 'Queue_name' 'Spot_Index' x y z rotation
    Queue_Spot Example_Queue 0 11 0 0 90

    Queue_Spot Example_Queue 1 9 0 0 0

    Queue_Spot Example_Queue 2 7 0 0 0

    Queue_Spot Example_Queue 3 5 0 0 0

After defining the queue, characters can be moved to the queue using the following:

command structure:

    # character goes to queue
    Character_01 Queue Example_Queue

    # action to perform when at 1st position in a queue
    Character_01 LookAround 10

    # destination to go to when leaving the queue. In this example, the character will go to 10, 10, 0.
    Character_01 Dequeue Example_Queue 10 10 0 _

Note

All three commands listed above are mandatory for the queue command to work correctly.

Custom Commands

You can create custom commands by providing animation USDs with the required attributes to our extension. When animation USDs are setup with the following attributes, they are identified as new command types and an animation graph node is created accordingly.

List of attributes to define an animation USD as custom command:

• CustomCommandName: The name of the custom command. Data type is string.

• CustomCommandTemplate: The command template used to register this animation. Data type is string.

• CustomCommandAnimStartTime: The animation start time. Data type is float.

• CustomCommandAnimEndTime: The animation end time. Data type is float.

There are three types of custom command templates:

• Timing: Plays the animation for a given time. The command structure is the same as Idle command.

• TimingToObject: Walks the character to an object and plays animation. The command structure is the same as Sit command.

• GoToBlend: Blends the animation into current walking animations. The command strcuture is the same as GoTo command.

Add a Custom Command

1. Open an animation USD, add all the attributes listed above and to the root prim. In this example, “CustomCommandName” is set to “CallingStand” and “CustomCommandTemplate” is set to “Timing”.

2. In PeopleSimulation window, under CustomCommand widget, click Add and select the animation USD in the last step. The extension is now able to recognize the new command.

3. Open an existing scene or follow the Getting Started section to create a new character scene, click on Setup Characters button to populate new animation graph node. Then use Command Text Box or Command Text File to verify the new command.

Note

Only the biped character under /World/Characters/Biped_Demo is populated with animation graph nodes.

Setup Simulation From Scratch

NavMesh defines the area that the characters can stand on and navigate to. It is essential for the random spawning function and command generation to work.

The omni.anim.navigation extension continues to evolve this process. For a tutorial, see the Omni.Anim.Navigation.

1. To add NavMesh Volume:

Right-click the viewport and select Create -> Navigation -> NavMeshVolume. The NavMeshVolume appears in the viewport as a box wireframe. Adjust its scale to cover the entire floor area, including props like tables and chairs, but exclude the ceiling.

2. Adjust Settings:

Open Window > Navigation > Navmesh. In the NavMesh panel, reset the defaults to appropriate values.

3. Bake the NavMesh from the NavMesh UI.

4. Disable the Show By Type > Navmesh to hide the navmesh.

5. After creating the navmesh, use the command text box or the command file shown in the examples above for setting up characters and creating a simulation.

Manually Setup Characters

We recommend populating characters through the UI of Omni.Anim.People.

1. To manually setup character, follow the steps below:

• Create a Xform under /World and rename it as Characters. It will be our character parent prim.

Note

[Isaac Sim Assets Path] is the path to Isaac Sim Assets. You need to set up Nucleus to use them. It can also be retrieved by printing omni.isaac.core.utils.nucleus.get_assets_root_path() with the Isaac Sim script editor.

• Create a Xform under /World and rename it as Characters. It is the character parent prim.

Note

To create an Xform with a specific name: right-click on the stage panel, then select Create > Xform. Double-click on the Xform prim and rename it to your desired name.

• From the Characters folder load the characters that you would like to use, by navigating to a characters folder and dragging and dropping the USD file into the /World/Characters Xform.

• Move the characters to their starting positions in the scene.

2. Setting up the characters:

• Find the behavior scripts provided with the extensions by navigating to Window > Extensions, searching for “people” and selecting omni.anim.people. Next, click on the folder icon to open the extension files.

• Navigate to \omni\anim\people\scripts and locate the character_behavior.py file.

• Expand each character on the stage menu and navigate until you find their SkelRoot. Example - /World/Characters/Character/male_adult_construction_01/ManRoot/male_adult_construction_01.

• Select the SkelRoot of every character by holding down control and clicking on each SkelRoot. Next navigate to SkelRoot > Right Click > Add > Animation > Animation Graph and select the only available animation graph.

• Select the SkelRoot of every character by holding down control and clicking on each SkelRoot. Select SkelRoot > Right Click > Add > Python Scripting to enable Python scripting.

• Go to the property window and in the Python Scripting property, select Add Asset and attach the character_behavior.py script.

3. Open up a text file and create commands for a simulation using the guide above. For characters that are manually setup, commands must be given using their Skelroot name. After the command file is ready, in the People Simulation set the command file in the Command File Path property.

4. Run the scene by clicking on Play.

Customize Assets and Scripts

Add Custom Behavior Scripts

You are encouraged to look at the character behavior scripts and modify them to fit your use case. To locate the Omni.Anim.People behavior scripts:

1. Select Window > Extensions to open the Extension Manager.

2. Search for “people”.

3. Click the folder icon to open the Extension files.

4. Navigate to \omni\anim\people\scripts.

To attach the modified scripts or any other character behavior scripts, follow the steps below:

1. Open Isaac Sim from the terminal by clicking on Open in Terminal Button.

2. In the console input:

   For Windows:

   isaac-sim.bat --/persistent/exts/omni.anim.people/behavior_script_settings/behavior_script_path="path to character behavior scripts"

   For Linux:

   isaac-sim.sh --/persistent/exts/omni.anim.people/behavior_script_settings/behavior_script_path="path to character behavior scripts"

The Setup Characters UI button now uses the provided character behavior scripts.

Adding Custom Character Assets

Add the new character USD file in the existing character assets folder ([Isaac Sim Assets Path]/Isaac/People/Characters), or create a new folder with the following structure:

Note

[Isaac Sim Assets Path] is the path to Isaac Sim Assets. You need to set up Nucleus to use them. It can also be retrieved by printing omni.isaac.core.utils.nucleus.get_assets_root_path() with the Isaac Sim script editor.

To set the ~character_assets_path~ value to point to the custom or modified character assets location:

1. Open Isaac Sim from the terminal, by clicking Open in Terminal Button.

2. In the console input:

   For Windows:

   isaac-sim.bat --/persistent/exts/omni.anim.people/asset_settings/character_assets_path="path to character asset root folder"

   For Linux:

   isaac-sim.sh --/persistent/exts/omni.anim.people/asset_settings/character_assets_path="path to character asset root folder"

The Load Characters UI button now uses the provided character asset location.

Note

New characters must be set up using Animation Retargeting.

Avoid Moving Objects

Characters can be made to avoid small and slow moving objects (like robots) by attaching the dynamic_obstacle.py script to the moving object. Avoidance is best effort and not guaranteed. Follow these steps:

• Find the ‘moving component’ of the object you want to avoid. This is generally the base prim of a physics articulation.

• On the prim, Right Click > Add > Python Scripting.

• In the property window, find the Python Scripting property, click on Select Asset and select the dynamic_obstacle.py script in the omni.anim.people extension files.

Note

Verify that the dynamic_obstacle.py on obstacle and the character_behavior.py on your characters are in the same folder. In addition, verify that the capitalization of your dynamic_obstacle.py file path are the same with the file path of your character_behavior.py.

Adding New Animations

You can add your own animations and actions to the simulation capabilities with a zero code process. Follow the steps below:

1. Open the animation graph, by navigating to Window > Animation > Animation Graph > Edit Animation Graph.

2. Double-click on the state machine to open it. Create a new state and add your animation in it.

3. Connect the state to the idle state with two transitions, from and to idle. Double click the transition arrows to modify the transitions. Right click and select Condition > ConditionCompareVariable. Attach the variable to the transition.

4. Open the ConditionCompareVariable in the property window and select the Action variable name. For the transition going to the idle node, this value must be None. For the transition going from the idle node, the value can be anything.

5. Go to the Script folder in the extension files and use the following command to generate a new Python command file. Note that the cmd_name must match the name of the node in the state machine and the transition_name must match the value used in the condition variable.

python cmd_file_generator.py -t .\commands\simple_cmd_template.txt -n cmd_name -tr transition_name

6. The above command generates a Python file with the name cmd_name.py. Drag and drop this file into the commands folder.

7. You can now use the command in the command file. For example,``character_name command duration``.

Troubleshooting

• In case an error is produced by the behavior scripts, the scripts need to be reinitialized by reloading the characters and setting them up again.

• In case a navigation-mesh is not generated after creating a navigation volume, open Window > Navigation > Navmesh. Review the values in NavMesh panel according to Omni.Anim.Navigation.

Note

In Isaac Sim, the default unit in the navmesh panel is meter instead of centimeter.

• To choose the characters that are loaded by the Omni.Anim.People UI, specify the character’s USD file name after the Spawn command. The character assets are present at [Isaac Sim Assets Path]/Isaac/People/Characters.

Note

[Isaac Sim Assets Path] is the path to Isaac Sim Assets. You need to set up Nucleus to use them. It can also be retrieved by printing omni.isaac.core.utils.nucleus.get_assets_root_path() with the Isaac Sim script editor.

• If the character does not walk on the ground plane, check the character’s retargeting setup. Adjust the facing direction by opening Window > Animation > Retargeting, and setting the Facing Directions in the Animation Retargeting panel correctly. In Isaac Sim, the value is typically, Forward: -Y, Up: Z.

Limitations

The extension is being released in its beta version and has the following observed limitations:

• Character dynamic avoidance is best effort and not guaranteed. In some scenarios where static and dynamic obstacles are present together, dynamic avoidance might cause erratic movement leading characters to walk into another character or into a static obstacle.

• Depending on the navigation mesh built, characters might go into static obstacles while walking. If this is observed, rebuild the navigation mesh with a higher agent radius.

• When constructing a queue, each queue spot must be at least 1 meter apart from each other.

• Character rotation in a fixed spot is done by rotating the whole prim.