Masking

Masking page contains all mixed reality masking related options.

• Type
  • Can be used to select which masking setup will be used to merge mixed reality image together with the VR image.
  • Depending on the selected type, other options for the specific masking setup will be available
  • Overlay
    • Places the mixed reality image in front of the VR image, so the mixed reality image will always be visible.
    • Depending on the used mixed reality resolution profile some parts of the VR image will be visible around the mixed reality image (as most mixed reality profiles will not cover the whole field of view of the headset).
    • No additional options are available for this masking type.
  • Background
    • Places the mixed reality image behind the VR image.
    • Requires using a compatible VR application with see-through parts of the virtual image (for example Unity vrghmd plugin with the ‘Alpha channel’ option active, or an OpenXR application utilizing the OpenXR alpha blend mode).
    • No additional options are available for this masking type.
  • 3D mesh
    • Uses a custom 3D mesh placed in the virtual space (relative to tracking origin) for mixed reality masking.
    • All parts of the 3D mesh will be filled with the mixed reality image, while the surroundings will be filled with the VR image.
    • 3D Mesh File
      • Shows path to the selected 3D mesh file.
      • If no 3D mesh file is selected, only the VR image will be shown inside the headset.
      • Clicking the Select button will allow you to select a custom 3D mesh file from your PC.
        • Only .obj or .fbx file formats are supported.
        • The 3D mesh file should only contain triangulated mesh objects, and it should not contain any material or texture information.
      • Clicking the Reload button will reload the currently selected 3D mesh file. Should be used if the 3D mesh file content was changed outside of the running VR application.
      • Clicking the Delete button will delete saved path to selected 3D mesh file (this will not delete the 3D mesh file from your PC).
    • Fill with Color
      • Can be used to fill the 3D mesh with specific color, in case that the whole 3D mesh is not covered by the mixed reality image.
      • This option can be useful if the selected mixed reality profile doesn’t cover the whole field of view of the headset.
      • If this option is disabled, any part of the 3D mesh which is not coveted by the mixed reality image will show the VR image instead.
      • Clicking the Set button will allow you to select a custom specific color for this functionality.
    • Show Mesh
      • Can be used to show a simple visualization of the currently used 3D mesh.
      • Mesh visualization will only be visible when the mixed reality itself is enabled, and it will replace the mixed reality image.
    • Wireframe
      • Can be used to show a combined visualization of the currently used 3D mesh and the mixed reality image.
      • Wireframe visualization will only be visible if the Show Mesh option is also enabled.
    • Reproject
      • Can be used to enable reproject functionality, which maps the mixed reality image onto the currently used 3D mesh.
      • This option will only work correctly if your 3D mesh very closely matches your real environment. Any real objects which are not a part of the 3D mesh will not look correct, as they will be fully mapped on the 3D mesh.
    • Match
      • Can be used to stabilize the mixed reality image relative to the virtual image.
      • This option is used to compensate for the different view point between the virtual scene and the mixed reality cameras.
      • We recommend leaving this option enabled for most use-cases.
    • Adjustment
      • Can be used to change positional and rotational offset of the 3D mesh (relative to tracking origin).
    • Scale
      • Can be used to change scale of the 3D mesh (either uniformly over all axes, or separately for each axis).
    • Tracking
      • Can be used to utilize a secondary SteamVR/Lighthouse tracker for 3D mesh position tracking.
      • When this option is enabled you can select Tracker ID from a list of all available SteamVR trackers. You can also use the Refresh IDs button to reload the list of SteamVR trackers.
      • The origin of the 3D mesh will then be relative to the position of the selected tracker, any Adjustment offsets will also apply.
  • Chroma key
    • Uses a specified color of the mixed reality image for chroma key masking.
    • Only parts of the mixed reality image which fit the specified color will show the VR image, other parts will only show the mixed reality image.
    • If the selected mixed reality resolution profile does not cover the whole field of view of the headset, some parts of the VR image will also be visible around the mixed reality image.
    • Color
      • Can be used to select a custom specific color for the chroma key masking.
      • Clicking the Load button will allow you to load a previously saved color from a file on your PC.
      • Clicking the Set button will allow you to select a custom color manually.
      • Clicking the Save button will save the currently used chroma key color into a .color file, so it can be loaded later.
    • Hue Tolerance
      • Can be used to adjust the hue tolerance of the chroma key algorithm.
      • If this option is set to higher values, the chroma key algorithm will use a bigger range of color hues around the specified color for masking.
    • Saturation Tolerance
      • Can be used to adjust the saturation tolerance of the chroma key algorithm.
      • If this option is set to higher values, the chroma key algorithm will use a bigger range of color saturations around the specified color for masking.
    • Value Tolerance
      • Can be used to adjust the value tolerance of the chroma key algorithm.
      • If this option is set to higher values, the chroma key algorithm will use a bigger range of color values around the specified color for masking.
    • Finetune
      • Can be used to tune the strength of the chroma key algorithm, in case that there is too much noise in the mixed reality image.
  • 3D mesh with chroma key
    • Uses a combination of the 3D mesh masking and chroma key masking.
    • The mixed reality image will only be shown in the area of the selected 3D mesh, on top of that this mixed reality image will be masked with the chroma key algorithm. So parts of the image fitting the specified color will show the VR image, even if they are in the area of the selected 3D mesh.
    • All additional options for 3D mesh masking and chroma key masking are also available for this masking type.
  • Cinema
    • Places the mixed reality image on a virtual 2D screen in a specified position in the virtual space (relative to the current virtual camera position).
    • While using this masking type the mixed reality image will not be adjusted for the current virtual eye point, so it may look physically incorrect depending on the angle and distance from the virtual 2D screen.
    • Horizontal Rotation
      • Can be used to change horizontal rotation of the virtual 2D screen.
    • Vertical Rotation
      • Can be used to change vertical rotation of the virtual 2D screen.
    • Distance
      • Can be used to change distance of the virtual 2D screen.
    • Reset
      • Allows resetting the rotation and distance options to their default values.
    • Headset Attached
      • Can be used to attach the virtual 2D screen to current headset position and rotation.
      • When this option is enabled the virtual 2D screen will move along with the headset movement, but the rotation and distance values will still apply.
    • Stereoscopic Video
      • Can be used to switch between using one single mixed reality image or two separate mixed reality images (stereoscopic).
      • When this option is enabled the virtual 2D screen will show image from left mixed reality camera for the left headset display, and image from the right mixed reality camera for the right headset display.
      • When this option is disabled only image from left mixed reality camera will be displayed on the virtual 2D screen.
    • Update Position
      • Updates the position of the 2D virtual screen relative to the current headset position.
  • Custom texture
    • Uses a custom texture (specified by the VR application) for masking.
    • Any area specified by the texture will show the mixed reality image, other areas will show the VR image.
    • Requires the VR application to submit separate mask textures for left and right eye (using the VRG_LAYER_LEFT_MR_MASK and VRG_LAYER_RIGHT_MR_MASK layer types).
    • No additional options are available for this masking type.
  • Custom texture (AND)
    • Uses a combination of the custom texture masking and chroma key masking.
    • The mixed reality image will only be shown in the area specified by the mask texture, on top of that this mixed reality image will be masked with the chroma key algorithm.
    • All additional options for chroma key masking are also available for this masking type.
  • Custom texture (XOR)
    • Uses a combination of the custom texture masking and chroma key masking.
    • The full mixed reality image will be shown in the area specified by the mask texture, outside of this area the mixed reality image will be also shown but masked with the chroma key algorithm.
    • All additional options for chroma key masking are also available for this masking type.

This page was last modified on April 7th, 2025 at 14:33

< PreviousNext >