Omniverse Radar Extension#

Introduction#

The Radar Sensor extension consists of RTX Sensor based models and omnigraph nodes for various post processing functions (ex: to transcode radar data in vendor specific formats)

The currently supported models are:

  • DmatApproxRadar (lower performance needs)

  • WpmDmatApproxRadar (higher fidelity)

Each of these Models has its own parameterization and at least one post processing omnigraph definition available that defines the nodes of the processing pipeline for the model with some default parameterization.

There can be multiple post processing omnigraphs for the same radar model, each can have its own parameter values and/or different nodes.

Under each model you can find details on parameterization.

Plugins#

DmatApproxRadarPlugin#

Introduction

The name is short for “Detection Matrix Approximation” radar model plugin. and it’s a model that emulates the behavior of real commercial radars by directly approximating the detection matrix or data cube that would be generated in these radars

The model aims to do this approximation using the Specs of the sensor normally provided in an OEM technical product specification. Since typically there is a huge number of parameters in a typical radar pipeline and often the processing stages in such a pipeline are OEM IP that can’t be shared, then approximating such pipelines with their specs can be advantageous.

This approximation aims to:

  • be real time.

  • be physically accurate to the extent needed for ADAS functions (ex: localization, fusion .. etc). it is not targeted for sensor validation or to be 100% matching to the real physical sensor.

  • capture important system-level effects (ex: merging detections at long range, resolution degradation with angle, cross talk between spatially different objects, multiple ghost detections .. etc.).

The output corresponds to simulating the radar as a black box, and producing raw radar detections.

Plugin Name

omni.sensors.nv.radar.dmatapprox.plugin

Model Parameters

The model supports 2 different scans (a near and far scan), each can have different parameterization. the full model params are listed in the table below

Parameter

Description

NearScanMaxRangeM

near scan max range in meters

NearScanMaxVelMpsSequence

near scan radial velocity sequences in meters per second

NearScanMaxAzAngRad

near scan max azimuth angle in radians (symmetry over bore sight implied)

NearScanMaxElevAngRad

near scan max elevation angle in radians (symmetry over bore sight implied)

NearScanRangeSepM

near scan range separation in meters

NearScanVelSepMps

near scan velocity separation in meters per second

NearScanBoreAzSepRad

near scan azimuth separation at bore sight in radians

NearScanBoreElevSepRad

near scan elevation separation at bore sight in radians

NearScanElevMode

near scan elevation mode (0 no elevation, 1 positive only, 2 full elevation)

FarScanMaxRangeM

far scan max range in meters

FarScanMaxVelMpsSequence

far scan radial velocity sequences in meters per second

FarScanMaxAzAngRad

far scan max azimuth angle in radians (symmetry over bore sight implied)

FarScanMaxElevAngRad

far scan max elevation angle in radians (symmetry over bore sight implied)

FarScanRangeSepM

far scan range separation in meters

FarScanVelSepMps

far scan velocity separation in meters per second

FarScanBoreAzSepRad

far scan azimuth separation at bore sight in radians

FarScanBoreElevSepRad

far scan elevation separation at bore sight in radians

FarScanElevMode

far scan elevation mode (0 no elevation, 1 positive only, 2 full elevation)

NumRaysPerDeg

num rays per degree

NumExtraThreads

num extra threads used (only relevant if Algo = 0)

FrameDurationNs

frame duration in nanoseconds

WpmDmatApproxRadarPlugin#

Introduction

The name is short for “Wave Propagation Model Detection Matrix Approximation” radar model plugin. The model uses the Wave Propagation Model (WPM) to enhance the fidelity compared to DmatApproxRadarPlugin, while still maintaining real time capability. New features are:

  • multiple bounces

  • support for different materials and improved radiometry

  • antenna gain patterns

Plugin Name

omni.sensors.nv.radar.wpm_dmatapprox.plugin

Model Parameters

The model supports multiple scans (e.g. near and far scan), each can have different parameterization. the full model params are listed in the table below

Parameter

Description

WaveLengthMm

Wave length in millimeters. Set to 3.9.

TraceTreeDepth

Number of bounces

FrameDurationNs

Duration of a frame in nanoseconds. Should match the sensor rate.

InstanceTimeOffsetUsec

Time offset for instances in microseconds.

CfarMode

Mode of CFAR: ‘2D’ for range-doppler. ‘4D’ for range-doppler-azimuth-elevation.

AntennaGainMode

Mode for antenna gain: ‘COSINEFALLOFF’ for a cosine falloff, ‘CONSTANT’ for constant gain over FOV.

ConfigReloadIntervalFrames

Number of frames after which configuration is reloaded, 0 for never. Currently not supported.

RaysPerDeg

Number of rays per degree in the scan. (Part of ScanCfgs)

TimeOffsetUsec

Time offset for this scan configuration in microseconds. (Part of ScanCfgs)

PowerFactor

Total output power of radar. (Part of ScanCfgs)

BinsFromSpec

Boolean flag to determine if bins are derived from specifications. (Part of ScanCfgs)

EnAngAliasing

Enable angular aliasing. (Part of ScanCfgs)

DetValFromBinIdx

‘true’: detection value derived from bin index ‘false’: more accurate inside cell (Part of ScanCfgs)

ElevMode

‘FULL_EL’ for full elevation tracking. ‘POS_EL’ only positive. ‘NO_EL’ no elevation.(Part of ScanCfgs)

MaxRangeM

Maximum scanning range in meters. (Part of ScanCfgs)

MaxAzAngDeg

Maximum azimuth angle in degrees. FOV is 2x MaxAzAngDeg (Part of ScanCfgs)

MaxElAngDeg

Maximum elevation angle in degrees. FOV is 2x MaxElzAngDeg (Part of ScanCfgs)

RangeResM

Range resolution in meters. Only works if BinsFromSpec is true. (Part of ScanCfgs)

VelResMps

Velocity resolution in meters per second. Only works if BinsFromSpec is true. (Part of ScanCfgs)

BoreAzResDeg

Azimuth resolution at bore sight in degrees. Only works if BinsFromSpec is true. (Part of ScanCfgs)

BoreElResDeg

Elevation resolution at bore sight in degrees. Only works if BinsFromSpec is true. (Part of ScanCfgs)

RBins

Number of range bins. Only works if BinsFromSpec is false. (Part of ScanCfgs)

VBins

Number of velocity bins. Only works if BinsFromSpec is false. (Part of ScanCfgs)

AzBins

Number of azimuth bins. Only works if BinsFromSpec is false. (Part of ScanCfgs)

ElBins

Number of elevation bins. Only works if BinsFromSpec is false. (Part of ScanCfgs)

CfarRnT

CFAR number of range test cells. (Part of ScanCfgs)

CfarRnG

CFAR number of range guard cells. (Part of ScanCfgs)

CfarVnT

CFAR number of velocity test cells. (Part of ScanCfgs)

CfarVnG

CFAR number of velocity guard cells. (Part of ScanCfgs)

CfarAznT

CFAR number of azimuth test cells. (Part of ScanCfgs)

CfarAznG

CFAR number of azimuth guard cells. (Part of ScanCfgs)

CfarElnT

CFAR number of elevation test cells. (Part of ScanCfgs)

CfarElnG

CFAR number of elevation guard cells. (Part of ScanCfgs)

CfarMinVal

CFAR minimum value for detection. Cells below that value will be discarded. (Part of ScanCfgs)

CfarOffset

Multiplied with raw CFAR noise level. Set to 1 for no offset (Part of ScanCfgs)

CfarNoiseMean

Mean noise level used in CFAR. Set to 0 for no mean noise. (Part of ScanCfgs)

CfarNoiseSDev

Standard deviation of noise in CFAR. Set to 0 for no gaussian noise.(Part of ScanCfgs)

MaxVelMpsSequence

Sequence of maximum unambiguous velocities in meters per second. (Part of ScanCfgs)

RcsTuningCoefficients

0:min threshold dBsm, 1:multiplier for rcs sm, 2:noise standard deviation dBsm. (Part of ScanCfgs)

AzimuthRadNoiseSDev

Standard deviation of azimuth radial noise in radians at bore sight. (Part of ScanCfgs)

RangeNoiseMean

Mean noise added to the range measurements. (Part of ScanCfgs)

RangeNoiseSDev

Standard deviation of noise added to the range measurements. (Part of ScanCfgs)

Available Parameterizations

The following parameterization to support specific radars already exists

Radar Model

Params File name

Generic

Example.json

Generic High Res

Example_highRes.json

The attributes are specified through a sensor config JSON file, which is placed in “data/sensors/radar/<senosr_type>”, where <sensor_type> is either dmat_approx_radar or wpm_dmat_approx_radar. Users can create custom parameterization files but they have to be placed in the same directory as the default parameterization files. The parameterization name is then the file name. This JSON file can be read with the ProfileReader utility plugin. The desired sensor profile is set via the USDA attribute “custom token rtxsensor:modelConfig”.

Parameterizing the model

Parameterization is specified through a sensor config JSON files, which can be found in “data/radar”. The desired param profile is set via the USDA attribute “custom token rtxsensor:modelConfig”. The parameterization name “Example” has to match one of the Available Parameterizations

Radar Point Cloud#

The Radar extension uses the GenericModelOutput as output format. See the documentation of the common extension for more information on the GenericModelOutput.

Omnigraph Nodes#

Currently there is only one node in the Radar extension. A transcoder which encodes the Radar data into vendor-specific UDP packets and dumps them to file.

TranscoderRadar#

Introduction

The Transcoder node encodes the radar data stream into vendor specific udp packets and dumps them to file. Currently, the node supports the following formats:

  • GENERIC

Omnigraph Node Name

omni.sensors.nv.radar.TranscoderRadar

Parameters & Attributes

Parameter Name

Description

Value Type

Value Range

Default Value

Example

remotePort

The port to address

int

10001

remoteIP

Remote IP of UDP channel

string

239.0.0.1

interfaceIP

Local interface to use

string

127.0.0.1

format

data send format

string

serviceID

ServiceMethod IDs in SOME/IP Header. Only for NCD formats

int

41610

fileName

Name for DW bin file

string

empty

Example Radar Pipeline#

def RenderProduct "RadarGeneric"
{
    def "PostRender"
    {
        def OmniGraph "DSPipeline"
        {

                custom token evaluator:type = "push"
                custom token flatCacheBacking = "StageWithoutHistory"
                custom token pipelineStage = "pipelineStagePostRender"


            def OmniGraphNode "RadarSource"
            {
                custom token node:type = "omni.graph.nodes.GpuInteropCudaEntry"
                custom int node:typeVersion = 1

                custom string inputs:sourceName = "RtxSensorCpu"
                custom uint64 outputs:cudaMipmappedArray
                custom uint64 outputs:format
                custom uint outputs:height
                custom uint outputs:mipCount
                custom uint64 outputs:stream
                custom uint outputs:width
                custom double outputs:simTime
                custom double outputs:hydraTime
            }

            def OmniGraphNode "RadarMetaSource"
            {
                custom token node:type = "omni.graph.nodes.GpuInteropCudaEntry"
                custom int node:typeVersion = 1

                custom string inputs:sourceName = "RtxSensorMetadata"
                custom uint64 outputs:cudaMipmappedArray
                custom uint64 outputs:format
                custom uint outputs:height
                custom uint outputs:mipCount
                custom uint64 outputs:stream
                custom uint outputs:width
                custom double outputs:simTime
                custom double outputs:hydraTime
            }

            def OmniGraphNode "Transcoder"
            {
                custom token node:type = "omni.drivesim.sensors.nv.radar.SinkPrepRadar"
                custom int node:typeVersion = 1

                custom uint64 inputs:cudaStream
                custom uint64 inputs:src
                custom uint64 inputs:meta
                custom int inputs:remotePort = 0

                custom uint64 outputs:cudaStream
                custom uint64 outputs:syncData
                custom uint64 outputs:taskingCounter
                custom uint64 outputs:dst
                custom uint64 outputs:bufferSize
                custom string outputs:channelName

                prepend uint64 inputs:src.connect = </Ecu/PostRender/DSPipeline/RadarSource.outputs:cudaMipmappedArray>
                prepend uint64 inputs:meta.connect = </Ecu/PostRender/DSPipeline/RadarMetaSource.outputs:cudaMipmappedArray>
                prepend uint64 inputs:cudaStream.connect = </Ecu/PostRender/DSPipeline/RadarSource.outputs:stream>
            }
        }
    }
}

Notes:

  1. in the above pipeline/graph definition there are only 2 nodes, a source node connected to a transcoder node that sends the sensor output as a UDP ethernet stream

  2. all RtxSensor post processing graphs has to start with the source node of type “omni.graph.nodes.GpuInteropCudaEntry” 1. for now the usda action can only override the “remotePort” with “action:camera:port” .. in case other processing graph params needs to be overridden a new graph definition needs to be built and referenced in the action

On this page