com.magicleap.mrtk3

Magic Leap MRTK3 Package

This package adds support for Magic Leap devices when using the Mixed Reality Toolkit 3 (MRTK3).

Current Status

Feature	Status
Controller	Pre-Release
Hand Tracking	Pre-Release
Eye Tracking	Pre-Release
Keyword Recognition	Pre-Release

Important Note

This package is an early access version. This means features are still in active development and subject to changes or even having their implementation completely removed and done in a different way.

Prerequisites

• Magic Leap SDK v1.2.0 (or later)
• Magic Leap Unity SDK v1.8.0 (or later)

Getting Started

Before importing the Magic Leap MRTK3 Early Access Package, developers will need to configure their project for MRTK3. This section provides general guidance on downloading and installing the MRTK3 packages using the Mixed Reality Feature tool (Windows Only) or using the MRTK3 Dev Template Project.

Using the MRTK Dev Template Project

Ready-made ML2 port of the MRTK Dev Template Project

There is a ready-made ML2 port of the MRTK Dev Template Project provided within the mrtk3_MagicLeap2 branch in the Magic Leap fork of the MRTK Github repository. This is the quickest and easiest way to get an MRTK3 app up and running on the Magic Leap 2. To use the ported Dev Template project, clone the forked MRTK GitHub repo and check out the mrtk3_MagicLeap2 branch. [Official Microsoft Guide Here] (https://learn.microsoft.com/en-us/windows/mixed-reality/mrtk-unity/mrtk3-overview/getting-started/setting-up/setup-template)

If you work with Git using the command line, you can clone the repo while specifying the mrtk3_MagicLeap2 branch: git clone --branch mrtk3_MagicLeap2 https://github.com/magicleap/MixedRealityToolkit-Unity

Original MRTK Dev Template Project

The original MRTK Dev Template project is available by downloading the MRTK project from the MRTK Github repository. To use the original Template project, clone MRTK from the GitHub repo and open the MRTKDevTemplate project under UnityProjects in Unity. [Official Microsoft Guide Here] (https://learn.microsoft.com/en-us/windows/mixed-reality/mrtk-unity/mrtk3-overview/getting-started/setting-up/setup-template)

Clone the repo on the command line: git clone https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity.git

Once the Github project is downloaded, follow the steps below to update the project settings to be compatible with Magic Leap 2.

Original Dev Template Project Setup

1. Using the Unity Hub, open the MRTKDevTemplate project using Unity version 2022.2.x or later.
   1. On the Opening Project in Non-Matching Editor Installation popup select Continue.
   2. On the Script Updating Consent popup select Yes, for these and other files that might be found later.
   3. On the Enter Safe Mode? popup select Ignore.
2. Clear any errors that appear as a result of a missing dependency from a prefab of XR provider.
   1. If Errors are still present, close the project and delete the project's Library folder and re-open unity to reimport the existing packages.
3. Download and Install the Magic Leap Setup Tool from the Unity Asset store.
4. Once installed, use the Project Setup window to configure your project settings. Complete all the steps in the project setup tool.

Creating a new MRTK3 Project

This section assumes you have already configured your Unity Project for ML2. (https://developer.magicleap.cloud/learn/docs/guides/unity/getting-started/configure-unity-settings)

Download the MRTK3 dependencies using one of two methods: Using the MixedReality Feature Tool (Windows Only), Expanded on below, or by manually downloading and importing the packages from Github. MRTK Input and MRTK UX Components packages are required for using with Magic Leap, please review MRTK Package Dependencies to know which dependencies are required for those two packages.

Using the Mixed Reality Feature Tool

This section provides instructions on installing the MRTK3 dependencies into an existing project using the Mixed Reality Feature tool. Note this tool is only available for Windows.

1. Open the Mixed Reality Feature tool
2. Target your Unity project.
3. At a minimum install the following packages, as they are required:

• MRTK3 / MRTK Input
• MRTK3 / MRTK UX Components

Note: If you do not see MRTK3, you may need to select the Show preview releases option located at the bottom of the window.

4. After choosing the the packages to install select Get Features. This will display the package dependencies.
5. Finally, press Import then Approve
6. Clear any errors that appear as a result of a missing dependency from a prefab of XR provider.

To learn more see Microsoft's Starting from a new project guide.

Importing the MRTK3 Magic Leap Package

Once the project is configured for ML2 and has the required MRTK3 packages, import the provided MRTK3 Magic Leap package into the project.

Open the Package Manager (Window > Package Manager) and import the com.magicleap.mrtk3.tgz package. Select the ＋ icon then select Add package from tarball...

Project Setup with XR Providers

There are currently two options of XR Providers that run on Magic Leap 2. Either, or both, XR Provider packages can be in the project, but only one active at a time. See the setup instructions below to configure your project to run with either.

Magic Leap XR Provider Setup

If the project contains the com.unity.xr.magicleap package, you can use the Magic Leap XR plugin. Follow the below steps to setup the MRTK3 project to run properly with this XR Provider.

1. Bring up project settings (Edit > Project Settings).

2. Within project settings, go to the XR Plug-in Management category on the left.

   1. Select the Android settings tab.
   2. Select the Magic Leap provider in the list. Make sure this is the only selected XR Provider for Android.

3. Within project settings, go to the MRTK3 category on the left.

   1. Set the Profile to MRTKProfile-MagicLeap, found at Packages/com.magicleap.mrtk3/Runtime/XRProviders/MagicLeap/Configuration/Default Profiles/.

4. Configure the MRTK XR Rig to be compatible with Magic Leap 2 input (2 provided options).

   1. First option: Use the Runtime MRTK XR Rig Configuration option in settings.
      • Within project settings, go to the MRTK3 > Magic Leap Settings category on the left.
      • Make sure the Magic Leap option is selected for Settings for XR Provider: at the top to view specific settings for the Magic Leap XR Provider.
      • Check the Runtime Rig Config Enabled checkbox at the top of the Runtime MRTK XR Rig Configuration window.
        • Enabling this feature will allow the default MRTK XR Rig to work with ML2 input without needing to modify the scene.
   2. Second option: In your scenes, remove and replace the default MRTK XR Rig with the MRTK XR Rig - MagicLeap prefab variant located in Packages/com.magicleap.mrtk3/Runtime/XRProviders/MagicLeap/Prefabs/MRTK_Variants/.

OpenXR XR Provider Setup

If the project contains the com.unity.xr.openxr package (version 1.9.1 or later), you can use the OpenXR XR plugin. Follow the below steps to setup the MRTK3 project to run properly with this XR Provider.

1. The following packages and their minimum versions are required to use OpenXR on Magic Leap 2 with MRTK3:

   • com.magicleap.unitysdk version 2.0.0 (or later)
   • com.unity.xr.openxr version 1.9.1 (or later)
   • com.unity.xr.hands version 1.4.0-pre.1 (or later)

2. Bring up project settings (Edit > Project Settings).

3. Within project settings, go to the XR Plug-in Management category on the left.

   1. Select the Android settings tab.
   2. Select the OpenXR provider in the list. Make sure this is the only selected XR Provider for Android.
      • Select the Magic Leap Feature Group.
   3. Select the XR Plug-in Management > OpenXR category on the left for more options.
      • Set the Depth Submission Mode to None.
      • Make sure the following interaction profiles are added:
        • Magic Leap 2 Controller Interaction Profile
        • Eye Gaze Interaction Profile
        • Hand Interaction Profile
      • In the feature list, make sure the Magic Leap 2 ... features you want are selected, plus Unity's Hand Tracking Subsystem feature.
      • Recommendation to change the OpenXR blend mode to Additive to avoid having Segmented Dimmer active by default. Select the Magic Leap 2 Rendering Extension settings icon. Set Blend Mode to Additive.
      • Note: If Microsoft's OpenXR package is also in the project, make sure Microsoft's Hand Tracking feature is not selected.

4. Within project settings, go to the MRTK3 category on the left.

   1. Set the Profile to MRTKProfile-MagicLeap-OpenXR, found at Packages/com.magicleap.mrtk3/Runtime/XRProviders/OpenXR/Configuration/Default Profiles/.

5. Configure the MRTK XR Rig to be compatible with Magic Leap 2 input (2 provided options).

   1. First option: Use the Runtime MRTK XR Rig Configuration option in settings.
      • Within project settings, go to the MRTK3 > Magic Leap Settings category on the left.
      • Make sure the OpenXR option is selected for Settings for XR Provider: at the top to view specific settings for the OpenXR XR Provider.
      • Check the Runtime Rig Config Enabled checkbox at the top of the Runtime MRTK XR Rig Configuration window.
        • Enabling this feature will allow the default MRTK XR Rig to work with ML2 input without needing to modify the scene.
   2. Second option: In your scenes, remove and replace the default MRTK XR Rig with the MRTK XR Rig - MagicLeap - OpenXR prefab variant located in Packages/com.magicleap.mrtk3/Runtime/XRProviders/OpenXR/Prefabs/MRTK_Variants/.

Magic Leap Permissions

The required Magic Leap permissions for your application must be added to the application's AndroidManifest.xml file. At a minimum for MRTK3 to use hand tracking, the HAND_TRACKING permission should be added to the manifest. This can be done in (Edit > Project Settings > Magic Leap > Permissions), if MLSDK is setup, or by adding it to the manifest file manually. Examples:

  <uses-permission android:name="com.magicleap.permission.HAND_TRACKING" />
  <uses-permission android:name="com.magicleap.permission.EYE_TRACKING" />

This package offers Runtime Permission Configuration in project settings to auto request and/or start certain permission easily without needing to add prefabs or code to your scenes to do so. Permission can be setup to be auto requested/started in (Edit > Project Settings > MRTK3 > Magic Leap Settings > Runtime Permissions Configuration).

For best eye tracking results, after having setup the EYE_TRACKING permission for your application, be sure to run the Custom Fit application and go through eye calibration.

Select any other permissions as needed for your application.

FAQ

Why do I need to use the Magic Leap rig variant or configure the rig at runtime? Replacing or configuring the rig ensures optimal bindings between Magic Leap inputs and the standard MRTK input action targets.