🕺Motion Capture

M² supports the use of real-time motion capture for VIPs, using LiveLink. This requires specialist equipment and setup - speak to your named engineer if you wish to use motion capture.

Motion Builder setup

The simplest solution is to Epic's LiveLink plugin for Motion Builder. Set this up as per the Unreal documentation. In Motion Builder take note of the Provider Name, and the Subject Name of the source skeleton data you wish to use.

Unreal setup

In Unreal you use M2_MocapCoordinatorComponent and M2M_SkeletalMocapReplicatorComponent to receive mocap data, retarget it to the player avatar and replicate the animation to other players.

Add a M2_MocapCoordinatorComponent to the pawn actor, and add a M2M_SkeletalMocapReplicatorComponent to the Morpheus actor. When mocap is enabled for the pawn on the local client it will listen for incoming mocap data and replicate it out, while enabling it on remote clients will cause them to listen for replicated animations being received by the Replicator component and update the local pose.

Connecting to LiveLink data

To connect using the Message Bus protocol (which is used by the Motion Builder plugin) you should call M2_MocapCoordinatorComponent::InitMocapMessageBus on every client when enabling mocap for that player. BusName should be the Provider Name specified in Motion Builder - this parameter is only used for the mocap broadcaster, and is ignored on remote clients.

Motion Builder should already have been configured to send data to the IP of broadcaster's machine.

Once connected, the actual transform data is pulled from the stream by the ReceiverSkeletalMesh animation blueprint (see below).

Client setup

Broadcasters must use a Test build of the client. This is because UDP messaging is disabled in Shipping builds for security reasons.

The broadcaster must also pass the -messaging command line argument when booting the client. Speak to your designated engineer to set up a Role for your broadcaster that will include this additional argument when using the Launcher.

M2_MocapCoordinatorComponent setup

Mocap data is a series of frames of named joint transforms. The simplest case is when this data uses the default UE5 skeleton joint names and hierarchy, however the component can be configured to support other formats. Use these properties on the component:

• ReceiverSkeletalMesh - This skeleton should match the joint hierarchy of the incoming mocap data. In unspecified it will use the pawn's skeleton. Incoming joint transforms will be applied directly to this skeleton by the MocapReceiverAnimClass.

• MocapReceiverAnimClass - This ABP is applied to the ReceiveSkeletalMesh. It will usually contain a LiveLinkPose node to read the LiveLink transform data. If joint names differ between the source data and the receiver skeleton, a RetargetAsset can be specified on the node to do name remapping. See ABP_M2_Human_Mocap_UE5Skel for a simple example where no remapping is required.

  
• RetargettingAnimClass - After receiving the data, the receiving skeleton is retargeted onto the pawn skeleton for display. Specify the ABP to use here. In the simple case where no retargeting is required, you can use ABP_M2_Human_MocapRetargeted_CopyOnly.

  

Debugging options

To assist with debugging issues with input data and retargeting, there are options in editor to show the various intermediate meshes and data. Set these options on the component:

• Show Mocap Receiver Mesh - This will render the ReceiverSkeletalMesh after it has had the mocap data applied.

• Show Retargeted Pose Mesh - This renders the mesh that will be replicated, after retargeting, on the broadcaster only.

• Show Replication Driven Mesh - This renders the replicated mesh, on remote clients only. These meshes are used as Leader Pose Components for the actual avatar mesh.

• Message Bus Debug Subject Name - Set this to the subject name to render a wireframe of the raw skeletal data for that subject.

• Message Bus Debug Draw Transform - An additional transform to apply to the debug skeleton, to correct for any rotation differences or to move it off to one side, for example.