Warping & Alignment

Screenberry offers a wide variety of methods for

• Geometry correction (warping)
• Transformations
• Soft-edge blending
• Projection alignment
• Mapping.

These methods are implemented with the help of the family of nodes called “patches” and other specialized nodes, which are described below.

The Rectangle node is a patch for applying planar rectangle-based modifications and is commonly used to scale, align and position the image on rectangular screens (mainly LED) or to create a picture-in-picture effect by combining multiple media sources on one screen.

In a typical setup, the Rectangle node receives a source image as the input and outputs the modified image to the Display or Canvas node.

Example of the node graph with the Rectangle node

The image geometry and mapping modifications with the Rectangle node are done in the Patch Editor:

• In the Geometry Layout Area, drag the handles of the mesh control points to change the position of the four vertices to determine the scale, aspect ratio, and location of the projected image.
• In the UV Layout Area, drag the handles of the mesh control points to change the scale, aspect ratio, and placement of the mapping image.

When editing the position of control points with the Rectangle patch, the additional white dotted diagonal smart guide is displayed. It corresponds to the aspect ratio of the source image based on its UV coordinates.

The QuadWarp node is a patch mostly used for image geometry correction which compensates for the distortion created by an off-axis image projection onto a planar surface (also known as keystone distortion).

In a typical setup, the QuadWarp node receives a source image as the input and outputs the modified image to the Display or Canvas node.

Example of the node graph with the QuadWarp node

Image geometry and mapping modifications with the QuadWarp node are done in the Patch Editor:

• In the Geometry Layout Area, drag the handles of the mesh control points to change the position of the four vertices that define the perspective of the projected image.
• In the UV Layout Area, drag the handles of the mesh control points to change the perspective of the mapping image.

The BezierWarp node is a patch that allows image warping using Bezier curves. It is mostly used for mapping a projection onto curved or partly curved surfaces and other regular or free-form objects.

In a typical setup, the BezierWarp node receives a source image as the input and outputs the modified image to the Display or Canvas node.

Example of the node graph with the BezierWarp node

Image geometry and mapping modifications with the BezierWarp node are done in the Patch Editor:

• In the Geometry Layout Area, drag the handles of the mesh control points to modify the warping mesh for the projected image.
• In the UV Layout Area, drag the handles of the mesh control points to adjust the mapping of the source image for the individual mesh cells.

Typically, warping with the BezierWarp starts from setting up the corner control points of the mesh according to the projection surface geometry, while the mesh density is set to the minimum. It’s followed by a gradual subdivision of the mesh grid while refining the curves of the mesh at each subdivision step until the desired image warping is achieved.

Another approach to editing the BezierWarp mesh is by selecting Interpolate Inner Points mode in the Settings panel. In this mode, the coordinates of inner control points are automatically recalculated to adapt to the modification of the Bezier curve caused by the movement of the patch grid edge points.

The QuadGrid node is a patch that applies perspective to the individual planes of the user-defined mesh grid. This patch is developed to map projection onto surfaces of stair-like objects.

In a typical setup, the QuadGrid node receives a source image as the input and outputs the modified image to the Display or Canvas node.

Example of the node graph with the QuadGrid node

To apply perspective warping:

1. Create a mesh grid by entering X and Y values in the Grid Size parameter in the Settings panel.

Typical settings for a projection onto a staircase-like object with horizontal planes (steps):

• The X value is 2 (the minimum number of key control points needed to define one column mesh grid). This value can be larger for projection mapping onto more complex objects.
• The Y value is equal to the total number of visible tread and riser surfaces.

For a projection on a staircase-like object with vertical planes, the X and Y values are reversed.

2. In the Geometry Layout Area, drag the handles of the mesh control points for each grid plane to modify the warping mesh for the projected image.
3. If the grid planes are transformed into trapezoidal shapes during modification of the mesh grid, some warping artifacts may occur at the junction of the planes. To avoid this, select the Linear option for the Vertical Interpolation or Horizontal Interpolation parameters in the Settings panel to limit perspective distortion to horizontal or vertical grid planes.

The TriangleMesh node is a patch that is mostly used to map projection onto the surfaces of objects that are composed of triangular shapes.

In a typical setup, the TriangleMesh node receives a source image as the input and outputs the modified image to the Display or Canvas node.

Example of the node graph with the TriangleMesh node

To apply triangular mesh warping:

1. Create and edit a triangular mesh by doing the following:

• Add control points. Position the cursor over the layout area, press the P key or left-click, and select Add Point in the drop-down menu to create a control point.
• Connect control points into triangular mesh faces. Select control points and press the T key, or left-click on the selection and choose Create Triangles in the drop-down menu to connect selected control points. If no control points are selected, pressing the T key connects all existing control points.
• Delete control points. Select control points and press the Del key or left-click on the selection, and choose Delete points in the drop-down menu to delete those control points.

2. In the Geometry Layout Area, drag the handles of the control points to modify the mesh grid.
3. In the UV Layout Area, drag the handles of the control points to define the mapping of the source image for the individual mesh triangles.

The XGrid node is a patch that is mostly used to map projections onto surfaces with an X-shaped grid pattern.

In a typical setup, the XGrid node receives a source image as the input and outputs the modified image to the Display or Context node.

Example of the node graph with the XGrid node

To apply X-grid mapping:

1. Create a mesh grid by entering X and Y values for the Grid Size parameter in the Settings panel.
   
2. In the Geometry Layout Area, drag the handles of the mesh control points for each grid cell to modify the warping mesh of the projected image.
3. In the UV Layout Area, drag the handles of the control points to define the mapping of the source image for the individual grid cells.

The PatchReplicator is a patch which duplicates and distributes the instances of the target patch according to user-defined rules. It is commonly used to create a picture-in-picture effect by combining multiple media sources on one screen.

Example of the node graph with the PatchReplicator node

There are three types of replication: Rectangular Array, Radial Array, and Mirror, which can be selected as a Type parameter in the Settings panel of the PatchesArray patch. Each type has different settings.

Rectangular Array replication

Using this replication, you can define the number of instances to be created and their offset along the X and / or Y axes.

When the Rectangular Array type is chosen the following replication parameters become available in the Settings panel:

• X Count and Y Count values that determine the number of instances to be created along the X and Y axes.
• X Offset and Y Offset values that determine the offset between instances relative to the entire width and height of the output area.

Radial Array replication

Using Radial Array replication, you can define the number of instances and their placement on a circle relative to the center of the output area (Сanvas). The most typical use of circular replication is placing copies of one image radially around dome zenith.

When the Radial Array type is chosen the following replication parameters become available in the Settings panel:

• A Count parameter value determines the number of instances to be created.
• An Angle parameter sets the angle of rotation.
• The Offset Mode options determine how the instances are distributed. If Between mode is selected, the angle between each of the created instances is equal to the value of the Angle parameter, and selection of Total allows to distribute instances at even angles within the angle specified in the Angle parameter.
• The Keep Aspect Ratio option specifies whether or not to maintain the aspect ratio for the created instances.

Mirror replication

Using mirroring, you can create a mirror copy of the image along the X or Y axes.

When the Mirror type is chosen, select the Horizontal or Vertical option for the Direction parameter to set the mirroring direction.

The SoftEdgeBlender node offers control over soft-edge blending for combining two or more overlapping projected images into a single seamless image.

The SoftEdgeBlender node generates blending maps to allow seamless overlapping for the images modified by patches and should be connected to the Blender input parameter of these patch nodes.

Example of the node graph with the SoftEdgeBlender node

The Settings panel of the SoftEdgeBlender node provides the following controls for soft-edge blending:

Blend Maps Size     Sets the dimensions of the blend map in pixels.

Generation Method    Select the blend map generation method: Linear Distance or Gravity Function.

Horizontal Overlap    Indicates that the blend maps are generated for the horizontal images overlap.

Vertical Overlap    Indicates that the blend maps are generated for the vertical images overlap.

Gravity Falloff    Sets the value for the gravitation-based edge detection algorithm for the blend maps generation if the Gravity Function blend map generation method is selected.

Quadratic Easing    Specifies if the quadratic easing function should be applied for the area of images overlap.

Blur Radius    Sets the blur radius for the blend maps.

Gamma    Sets the inverse-gamma distribution value for the blend maps.

Build Blend Maps    Pressing the Build Blend Maps button generates new blend maps.

Disable Blending    Resets the blend maps.

The Transform2D node allows you to apply static or dynamic post-processing transformations to the patches. Typically, it is used to position, move, scale, and rotate patches and create various picture-in-picture effects.

The Transform2D node connects to the Transform Matrix input parameter of the target patch node.

Example of the node graph with the Transform2D node

The Settings panel of the Transform2D node offers control over the following transformation parameters:

Translate    Offsets the patch horizontally (X) and/or vertically (Y) by the set factor.

Rotate    Rotates the patch around its center by the specified angle.

Scale    Scales the patch from its center by the set factor.

Transform Order   Select an order in which transformations are implemented, for example, Scale, Rotate, Translate.

Flip V    Flips the patch vertically.

Flip H    Flips the patch horizontally.

Aspect Ratio    Stretches the patch horizontally by the set factor.

Translation Speed    Sets the speed for continuous horizontal (X) and/or vertical (Y) movement of the patch.

Scaling Speed    Sets the speed for continuous scaling of the patch from its center.

Rotation Speed    Sets the speed for continuous rotation of the patch around its center by the specified angle.

Reset to Default    Resets all transformations applied to the patch by the Transform2D node.

The DomeTransform node is a patch that allows the application of additional geometry correction to the image aligned by automatic calibration for the dome setup. Typically, it is used to compensate for the position, orientation, and lens distortion of the calibration camera.

Generally, the DomeTransform node connects to the Calibrator node via the RenderTarget node.

Example of the node graph with the DomeTransform node

When the DomeTransform node is added, the DomeTransform Controls widget becomes available in the list of widgets.

Both the Settings panel of the DomeTransform node and the DomeTransform Controls widget offer control over the following transformation parameters: 

Grid Size    Sets the number of rows and columns for the patch mesh grid.

Subdivisions    Sets the number of subdivisions for the patch mesh grid. 

Rotation Angle    Rotates the projection image by the set angle.

Flip V    Flips the projection image vertically.

FOV    Changes the field of view angle by the set factor.

Lens Distortion    Compensates for lens distortion of the calibration camera. This parameter transitions through the following mapping functions of the lens: Equidistant at 0, Equisolid angle for values closer to 0.5 (1/k2 value), and Orthographic at 1.
For more information, visit https://en.wikipedia.org/wiki/Fisheye_lens#Mapping_function.

Tilt Angle    Compensates for the calibration camera tilt.

Offset X    Compensates for the horizontal shift of the calibration camera position.

Offset Y    Compensates for the vertical shift of the calibration camera position.

Offset Z    Compensates for the calibration camera being positioned closer or farther away from the zenith of the dome.

The PlanarToDome node is a patch that allows mapping of a planar image onto a dome surface. It is mostly used to achieve a picture-in-picture effect by combining planar and dome media content on a dome screen.

In a typical setup, the PlanarToDome node receives a source image as the input and outputs the modified image to the Display or Context node.

Example of the node graph with the PlanarToDome node

The Settings panel of the PlanarToDome node offers the following geometry correction parameters:

Grid Size    Sets the number of rows and columns for the patch mesh grid.

Subdivisions    Sets the number of subdivisions for the patch mesh grid.

Aspect Ratio   Defines the aspect ratio of the planar media. 

Projection   Allows you to select the type of three-dimensional shape on which the image is mapped by choosing one of the following options: Planar, Conical, or Cylindrical.

Scale   Scales the image arbitrary from its center by the set factor.

Vertical Offset   Offsets the vertical position of the image in relation to the dome zenith.

Tilt Angle   Sets the angle of the image tilt.

Rotation Angle   Rotates the image by the specified angle around the zenith of the dome.

The LatLongToDome node is a patch that allows the mapping of VR 360-degree media onto a virtual sphere and provides controls for displaying it on a dome surface.

In a typical setup, the LatLongToDome node receives a source image as the input and outputs the modified image to the Display or Canvas node.

Example of the node graph with the LatLongToDome node

The Settings panel of the LatLongToDome node offers the following parameters for geometry correction and control over the media presentation:

Initial Orientation   Allows you to define the default orientation of the media after the patch reset or server restart. X and Y values set the longitude and latitude rotation angles. The Y value has set extremes at 90° and -90° which correspond to the zenith and nadir of the virtual sphere.

Current Orientation   Allows you to edit the current position of the media. X and Y values set the longitude and latitude rotation angles. Editing of this parameter is enabled by selecting the Manual Position option for the Position Input parameter.

Rotation Speed    Sets the angle for the step of constant media rotation along X– or Y-axes. This parameter is disabled by default. To enable editing of this parameter, select the Rotation Speed for the Position Input parameter.

Speed Multiplier    Sets the sensitivity level (the rotation angle step size multiplier) when rotation is controlled by external input parameters.

Lat. Segments    Sets the number of the latitude segments for the virtual sphere to which the media is mapped.

Long. Segments    Sets the number of longitude segments for the virtual sphere to which the media is mapped.

Camera Offset Z    Offsets the position of the virtual camera in relation to the dome zenith.

Position Input    Allows you to select the type of control for media orientation by choosing the Manual Position option for manual input, or the Rotation Speed option when media orientation is controlled by external input parameters.

Camera FOV    Sets the fisheye lens field of view angle for the media.

Visible FOV    Sets the angle of masking for the fisheye lens field of view.

The LedStripesMapper node is a patch that allows the remapping of media content into lines. Typically, it is used to output the signal to a LED stripes controller. It can also be used to control lighting devices that can receive video as input.

Typically, the LedStripesMapper node receives the media content from the MediaPlayer node and sends output to the Canvas node.

Example of the node graph with the LedStripesMapper node

To remap the content, do the following:

1. In the Node Graph tab, create a node tree by adding the MediaPlayer,  LedStripesMapper, and Canvas nodes, and connect them by Ctrl-clicking them one by one.
2. Switch to the Media Player tab, add the media content you want to remap into the lines to the MediaPlayer playlist, and start playback to have a visual reference while working with the patch.

3. Switch to the Canvas Editor tab, and select the LedStripesMapper in the patches list.

4. In the Settings panel of the LedStripesMapper:

• In the Source Line Width parameter, set the width in pixels of the source media to be remapped as lines.
• Enter values for the Target Image Size, Offset X and Y, output Line Width, and Spacing parameters (consult your LED stripes or lighting controller documentation for the format details).
• Enter a value for the Number of Polylines parameter, and for each polyline enter the number of Segments and Total Length in pixels (for example, you have a polyline with 5 sections and each section is 128 pixels, then the total length is 5 x 128 = 640 pixels).

5. In the UV Layout Area, press the F5 key to load a snapshot of the source media, then map the polylines one by one by dragging the control points over the polyline endpoints.