Warping & Alignment

Overview

Screenberry offers a wide variety of methods for

  • Geometry correction (warping)
  • Transformations
  • Soft-edge blending
  • Projection alignment
  • Mapping.

These methods are implemented with the help of the family of nodes called “patches” and other specialized nodes, which are listed below.

Note:
In addition to the features specific to each of the patch nodes described below, the general aspects of working with the patches are also applicable (for more information, see Editing Patches).

Rect

The Rect node is a patch for applying planar rectangle-based modifications and is commonly used to scale, align and position the image on rectangular screens (mainly LED) or to create a picture-in-picture effect by combining multiple media sources on one screen.

In a typical setup, the Rect node receives a source image as the input and outputs the modified image to the Display or Context node.

Example of the node graph with the Rect node

The image geometry and mapping modifications with the Rect node are done in the Patch Editor:

  • In the Geometry Layout Area, drag the handles of the mesh control points to change the position of the four vertices to determine the scale, aspect ratio, and location of the projected image.
  • In the UV Layout Area, drag the handles of the mesh control points to change the scale, aspect ratio, and placement of the mapping image.

When editing the position of control points with the Rect patch, the additional white dotted diagonal smart guide, which corresponds to the aspect ratio of the source image based on its UV coordinates, is displayed.

PerspQuad

The PerspQuad node is a patch mostly used for image geometry correction which compensates for the distortion created by an off-axis image projection onto a planar surface (also known as keystone distortion).

In a typical setup, the PerspQuad node receives a source image as the input and outputs the modified image to the Display or Context node.

Example of the node graph with the PerspQuad node

Image geometry and mapping modifications with the PerspQuad node are done in the Patch Editor:

  • In the Geometry Layout Area, drag the handles of the mesh control points to change the position of the four vertices that define the perspective of the projected image.
  • In the UV Layout Area, drag the handles of the mesh control points to change the perspective of the mapping image.

BezierPatch

The BezierPatch node is a patch that allows image warping using Bezier curves. It is mostly used for mapping a projection onto curved or partly curved surfaces and other regular or free-form objects.

In a typical setup, the BezierPatch node receives a source image as the input and outputs the modified image to the Display or Context node.

Example of the node graph with the BezierPatch node

Image geometry and mapping modifications with the BezierPatch node are done in the Patch Editor:

  • In the Geometry Layout Area, drag the handles of the mesh control points to modify the warping mesh for the projected image.
  • In the UV Layout Area, drag the handles of the mesh control points to adjust the mapping of the source image for the individual mesh cells.

Typically, warping with the BezierPatch starts from setting up the corner control points of the mesh according to the projection surface geometry, while the mesh is set to the minimum density. It’s followed by a gradual subdivision of the mesh grid while refining the curves of the mesh at each subdivision step until the desired image warping is achieved.

Another approach to editing the BezierPatch mesh is by selecting Auto adjust internals mode in the Settings panel. In this mode, the coordinates of inner control points are automatically recalculated to adapt to the modification of the Bezier curve caused by the movement of the patch grid edge points.

PerspQuadArray

The PerspQuadArray node is a patch that applies perspective to the individual planes of the user-defined mesh grid. This patch is developed to map projection onto surfaces of stair-like objects.

In a typical setup, the PerspQuadArray node receives a source image as the input and outputs the modified image to the Display or Context node.

Example of the node graph with the PerspQuadArray node

To apply perspective warping:

  1. Create a mesh grid by entering X and Y values in the Grid Size parameter in the Settings panel.

Typical settings for a projection onto a staircase-like object with horizontal planes (steps):

  • The X value is 2 (the minimum number of key control points needed to define one column mesh grid). This value can be larger for projection mapping onto more complex objects.
  • The Y value is equal to the total number of visible tread and riser surfaces.

For a projection on a staircase-like object with vertical planes, the X and Y values are reversed.

Example of a patch grid for an object with horizontal planes. Grid Size: X=2, Y=10.

Example of a patch grid for an object with vertical planes. Grid Size: X=10, Y=2.

  1. In the Geometry Layout Area, drag the handles of the mesh control points for each grid plane to modify the warping mesh for the projected image.
  2. If the grid planes are transformed into a trapezoidal shape during modification of the mesh grid, some warping artifacts may occur at the junction of the planes. To avoid this, select the horizontalStairs or verticalStairs checkboxes in the Settings panel to limit perspective distortion to horizontal or vertical grid planes.

 

TriMesh

The TriMesh node is a patch that is mostly used to map projection onto the surfaces of objects that are composed of triangular shapes.

In a typical setup, the TriMesh node receives a source image as the input and outputs the modified image to the Display or Context node.

Example of the node graph with the TriMesh node

To apply triangular mesh warping:

  1. Create a triangular mesh by doing the following:
  • Add control points. Position the cursor over the layout area, press the P key or left-click, and select Add Point in the drop-down menu to create a control point.
  • Connect control points into triangular mesh faces. Select control points and press the T key, or left-click on the selection and choose Create Triangle(s) in the drop-down menu to connect selected control points. If no control points are selected, pressing the T key connects all existing control points.
  • Delete control points. Select control points and press the Del key or left-click on the selection, and choose Delete point(s) in the drop-down menu to delete those control points.
  1. In the Geometry Layout Area, drag the handles of the control points to modify the mesh grid.
  2. In the UV Layout Area, drag the handles of the control points to define the mapping of the source image for the individual mesh triangles.

XGrid

The XGrid node is a patch that is mostly used to map projections onto surfaces with an X-shaped grid pattern.

In a typical setup, the XGrid node receives a source image as the input and outputs the modified image to the Display or Context node.

Example of the node graph with the XGrid node

To apply X-grid mapping:

  1. Create a mesh grid by entering X and Y values for the Grid Size parameter in the Settings panel.
  2. In the Geometry Layout Area, drag the handles of the mesh control points for each grid cell to modify the warping mesh of the projected image.
  3. In the UV Layout Area, drag the handles of the control points to define the mapping of the source image for the individual grid cells.

PatchesArray

The PatchesArray is a patch which duplicates and distributes the instances of the target patch according to user-defined rules. It is commonly used to create a picture-in-picture effect by combining multiple media sources on one screen.

Note:
The PatchesArray node can receive image input from the patch nodes only.

Example of the node graph with the PatchesArray node

There are three types of replication: Linear, Mirror, and Circle, which can be selected as a Type parameter in the Settings panel of the PatchesArray patch. Each type has different settings.

Linear replication

Using linear replication, you can define the number of instances to be created and their offset along the X and / or Y axes.

When the Linear type is chosen the following replication parameters become available in the Settings panel:

  • Forward or Reverse options for the X Direction and Y Direction parameters to specify the direction of replication.
  • X Number and Y Number values that determine the number of instances to be created along the X and Y axes.
  • X Step and Y Step values that determine the offset between instances relative to the entire width and height of the output area.

Mirroring replication

Using mirroring, you can create a mirror copy of the image along the X or Y axes.

When the Mirror type is chosen, select the X-Axis or Y-Axis option for the Mirror Axis parameters.

Circular replication

Using Circular replication, you can define the number of instances and their placement on a circle relative to the center of the output canvas (Context). The most typical use of circular replication is placing copies of one image radially around dome zenith.

When the Circle type is chosen the following replication parameters become available in the Settings panel:

  • Clockwise / Anticlockwise options for the Direction parameter to specify the direction of the replication rotation.
  • A Quantity value to determine the number of instances to be created.
  • Instance Spacing / Equal Spacing options for the Circle Array Mode parameter.

An Angle parameter. If Instance Spacing mode is selected, the angle of each instance rotation is equal to the value of the Angle parameter. If Equal Spacing mode is selected, the angle of each instance rotation is equal to the value of the Angle parameter divided by the number of instances.

Blender

The Blender node offers control over soft-edge blending for combining two or more overlapping projected images into a single seamless image.

The Blender node connects to the Blender input parameter of the patch nodes and generates blending maps for the images modified by the patch nodes.

Example of the node graph with the Blender node

The Settings panel of the Blender node provides the following controls for soft-edge blending:

BlendMaps Size    Sets the dimensions of the blend map in pixels.

Horizontal Panorama    Indicates that the blend maps are generated for the horizontal images overlap.

Vertical Panorama    Indicates that the blend maps are generated for the vertical images overlap.

Apply quadratic easing    Specifies if the quadratic easing function should be applied for the area of images overlap.

Maps build method    Select the blend map generation method: Simple distance transform or Advanced gravity function.

Gamma value     Sets the inverse-gamma distribution value for blend maps.

Gravity power value    Sets the value for the gravitation-based edge detection algorithm for the blend maps generation if the Advanced gravity function is selected.

Blur radius    Sets the blur radius for the blend maps.

Disable blendMaps    Temporarily disables soft edge blending.

Disable blends    Resets the blend maps.

Build blends    Generates new blend maps by pressing the Build blends button.

Note:
Regenerate blend maps by pressing the Build blends button every time that the settings of the Blender node are changed or when the geometry is modified in patch nodes connected to the Blender node.

Transform

The Transform node allows you to apply static or dynamic post-processing transformations to the patches. Typically, it is used to position, move, scale, and rotate patches and create various picture-in-picture effects.

The Transform node connects to the Transform input parameter of the target patch node.

Example of the node graph with the Transform node

Note:
Transformations applied by the Transform node do not affect the settings and geometry of the target patch node.

The Settings panel of the Transform node offers control over the following transformation parameters:

translate    Offsets the patch horizontally (X) and / or vertically (Y) by the set factor.

scale    Scales the patch from its center by the set factor.

rotate    Rotates the patch around its center by the specified angle.

aspectRatio    Stretches the patch horizontally by the set factor.

flipH    Flips the patch horizontally.

flipV    Flips the patch vertically.

Transformation order    Select an order in which transformations are implemented, for example, Scale, Rotate, Translate.

constantMove    Continuously moves the patch horizontally (X) and / or vertically (Y) by the set factor.

constantScale    Continuously scales the patch from its center by the set factor.

constantRotate   Continuously rotates the patch around its center by the specified angle.

Reset    Resets all transformations applied to the patch by the Transform node.

DomeTransform

The DomeTransform node is a patch that allows the application of additional geometry correction to the image aligned by automatic calibration for the dome setup (link). Typically, it is used to compensate for the position, orientation, and lens distortion of the calibration camera.

Generally, the DomeTransform node connects to the Calibrator node via the RenderTarget node.

Example of the node graph with the DomeTransform node

When the DomeTransform node is added, the DomeTransform Controls widget becomes available. You can access it in the Media Player tab by selecting the Window / DomeTransform Controls checkbox.

Settings panel of the DomeTransform node

DomeTransform Controls widget

Both the Settings panel of the Transform node and the DomeTransform Controls widget offer control over the following transformation parameters: 

Grid Size    Sets the number of rows and columns for the patch mesh grid.

Subdivisions    Sets the number of subdivisions for the patch mesh grid. 

DomeTransform Controls widget offers simpler controls for the same parameters

rotation    Rotates the projection image by the set angle.

flip    Flips the projection image vertically.

FOV    Changes the field of view angle by the set factor.

fisheyeLenseDistortion    Compensates for the lens distortion of the calibration camera. Depending on the set value, this parameter transitions through the following mapping functions of the lens: Equidistant for higher negative values, Equisolid angle for values closer to zero, and Orthographic for higher positive values.

tiltVertical    Compensates for the calibration camera tilt.

cameraPositionX    Compensates for the horizontal shift of the calibration camera position. 

cameraPositionY    Compensates for the vertical shift of the calibration camera position.

cameraPositionZ    Compensates for the calibration camera being positioned closer or farther away from the zenith of the dome.

PlanarToDome

The PlanarToDome node is a patch that allows mapping of a planar image onto a dome surface. It is mostly used to achieve a picture-in-picture effect by combining planar and dome media content on a dome screen.

In a typical setup, the PlanarToDome node receives a source image as the input and outputs the modified image to the Display or Context node.

Example of the node graph with the PlanarToDome node

The Settings panel of the PlanarToDome node offers the following geometry correction parameters:

Grid Size    Sets the number of rows and columns for the patch mesh grid.

Subdivisions    Sets the number of subdivisions for the patch mesh grid.

Aspect Ratio   Defines the aspect ratio of the planar media. 

Type   Allows you to select the type of three-dimensional shape on which the image is mapped (cylinder, cone, or sphere) by choosing one of the following options: Flat, Curved, or Curved(Parallel).

Size   Scales the image arbitrary from its center by the set factor.

Height   Sets the distance from the image to the dome zenith.

Tilt   Sets the angle of the image tilt.

Rotation   Rotates the image by the specified angle around the zenith of the dome.

PanoToDome

The PanoToDome node is a patch that allows the mapping of VR 360-degree media onto a virtual sphere and provides controls for displaying it on a dome surface.

In a typical setup, the PanoToDome node receives a source image as the input and outputs the modified image to the Display or Context node.

Example of the node graph with the PanoToDome node

The Settings panel of the PanoToDome node offers the following parameters for geometry correction and control over the media presentation:

initialPosition   Allows you to define the default position of the media after the patch reset or server restart. X and Y values set the longitude and latitude rotation angles. The Y value has set extremes at 90° and -90° which correspond to the zenith and nadir of the virtual sphere.

position    Allows you to edit the current position of the media. X and Y values set the longitude and latitude rotation angles. This parameter is disabled by default. To enable editing of this parameter select the useInputPosition checkbox.

rotation    Sets the angle of the constant rotation of the media along X– or Y-axes.

rotationSensitivity    Sets the sensitivity level (the rotation angle step size) when rotation is controlled.

sides    Sets the number of the longitude segments for the virtual sphere to which the media is mapped.

segments    Sets the number of the latitude segments for the virtual sphere to which the media is mapped.

cameraBackOffset    Changes the distance from the virtual camera to the zenith of the dome.

useInputPosition    Select this checkbox to allow editing of position parameters manually. 

output fisheye FOV, deg     Sets the fisheye lens field of view angle for the media.

output fisheye FOV visible, deg     Sets the angle of masking for the fisheye lens field of view.

Note:
When the fisheye lens field of view angle approaches 360°, some artifacts may appear on the edges of the image. To avoid this, use a mask to limit the visible part of the image.

LinesPatch

The LinesPatch node is a patch that allows the remapping of media content into lines. Typically, it is used to output the signal to a LED stripes controller. It can also be used to control lighting devices that can receive video as input.

Typically, the LinesPatch node receives the media content from the MediaPlayer node and sends output to the Context node.

Example of the node graph with the LinesPatch node

To remap the content, do the following:

  1. In the Node Graph tab, create a node tree by adding the MediaPlayerLinesPatch, and Context nodes, and connect them by Ctrl-clicking them one by one.
  2. Switch to the Media Player tab, add the media content you want to remap into the lines to the MediaPlayer playlist, and start playback to have a visual reference while working with the patch.
  1. Switch to the Patch Editor tab, and select the LinesPatch in the patches list.
  1. In the Settings panel of the LinesPatch:
  • In the Source line width parameter, set the width in pixels of the source media to be remapped as lines.
  • Enter the Output size, Output starting point X and Y coordinates, Output line width, and Output rows clearance parameters (consult your LED stripes or lighting controller documentation for the format details).
  • Enter the number of polylines in the Polylines count parameter, and for each polyline enter the number of endpoints and output pixels (for example, you have a polyline with 5 sections and each section is 128 pixels, then the number of output pixels is 5 x 128 = 640 pixels).

5. In the UV Layout Area, press the F5 key to load a snapshot of the source media, then map the polylines one by one by dragging the control point over the polyline endpoints.