Display Walls

CommercialBeta

Included in the public demo Compare editions

A DisplayWall turns an LED wall, a powerwall or a projector into a window onto the twin. You measure the physical wall once. The viewer then renders the model so that it appears to stand behind that window, from each viewer’s tracked eye position. One output canvas can carry a stereo pair for one person, stereo pairs for two people, or separate views for up to eight areas.

Display walls, stereo output and Flystick control are part of realvirtual Commercial as a Beta (since 6.6.0).

What you need

Part	Purpose
Output PC with Chrome or Edge	Runs the viewer in fullscreen on the wall’s input; WebGL required
Wall, TV or projector	Receives one image at the configured canvas size and shows its areas to the right eyes or people
Stereo glasses	Only for stereo layouts; their type follows the display, not the viewer
Tracking system (optional)	ART DTrack or OptiTrack Motive, received by CONNECT; without tracking every area uses a fixed SweetSpot
ART Flystick (optional)	Pointing, selecting, moving parts and flying through the model; DTrack only
SpaceMouse (optional)	Navigation at the wall; see 3D mouse (SpaceMouse)

The viewer draws all areas into one browser canvas, by default 3840 × 2160 px. It does not output frame-sequential (quad-buffer) stereo itself. The wall controller, LED processor or projector must take the eye images as areas of one input image and present them in its own stereo mode, for example side by side or in quadrants. Ask the display supplier which input layout and resolution the controller expects for each eye, then match the viewports to it.

Refresh rate and load

An internal measurement on the demo model gives a starting point. A Ryzen 9 9950X with an RTX 5060 Ti rendered four viewports into a 3840 × 2160 px buffer at 105 FPS without VSync. The median frame time was 9.2 ms, and the 95th percentile 11.9 ms. Rendering was CPU-bound, and each additional area added about 2 ms.

Target	Result on the measured machine
60 Hz, four areas, 4K buffer	Reached, with about 7.5 ms median reserve per frame
120 Hz active stereo, four areas	Not reached; one frame may take at most 8.3 ms
Two areas (one stereo pair), buffer of about 1080p	195 FPS

The test used a simulated fullscreen adapter, not physical 4K wall output. Use a discrete midrange GPU and a CPU with strong single-thread performance as a starting point. Measure your own model on the output PC, especially with larger models or more than four areas.

Create the wall

The wall is a DisplayWall component on a node in the project or layout document, so it is saved and delivered with the project. Each output area is a WallViewport and each tracked controller a WallFlystick, both as child nodes of the wall. Create the wall in one of two ways:

Settings ▸ Stereo 3D ▸ Display wall — open the project in the editor, choose a Wall template and, optionally, a Tracking source (the CONNECT interface, for example dtrack), then click Create display wall. This adds a root node Display wall with the template’s viewports as one undoable step. With a tracking source, every viewport’s signal slots are filled from its body (see Connect head tracking). Signals that do not exist yet are listed under Not found; the wall is created anyway and those slots stay empty. Outside the editor, or while the document is busy with another edit, the button is disabled and shows the reason.
Inspector — add a DisplayWall component to any node of your project or layout document, then add viewports and Flysticks in its editor.

Once a wall exists, the Display wall section of the Stereo 3D tab shows the same editor as the inspector, the wall’s state (Off, Requested or Active), the tracking status of each viewport and the start and stop buttons. Show in inspector closes the settings and selects the wall node. Layout locks apply here as in the inspector. With several walls in the scene, choose one under Wall.

Measure the wall

The node places the wall in the model; the corner coordinates describe the physical wall in the tracking coordinate system.

Measure the lower-left, lower-right and upper-left corners of the visible image area as seen from the audience side. Enter CornerLowerLeft, CornerLowerRight and CornerUpperLeft in millimetres, using CONNECT’s right-handed, Y-up tracking coordinates.
Check that the two wall edges form a right angle within ±2°. Collinear corners and invalid values are marked in the inspector and prevent output from starting.
Set ModelScale to the model-to-physical-space ratio 1:N. The default is 1; increasing N makes the measured wall and head movements span more model space. The allowed range is 0.001–10000.
Set CanvasWidth and CanvasHeight to the wall controller’s input resolution. Defaults are 3840 × 2160 px; each dimension accepts whole numbers from 320 to 16384.
Set SweetSpot to a fallback head position in the same tracking coordinates. Its default is (0, 1350, 2500) mm. It is a stored position, so update it when you change the measured wall.

The example corner defaults are (-2400, 0, 0), (2400, 0, 0) and (-2400, 2700, 0) mm. Replace these with your measurements. Near is the minimum eye distance from the wall plane, default 0.05 m, range 0.01–1 m.

Choose the output areas

Each WallViewport projects the full measured wall from its assigned eye into its own rectangle of the canvas. The wall draws its viewports in the order of its child nodes. The templates replace the wall’s viewport children; the canvas size stays as configured.

Template	Viewports	Filled from (with a tracking source)
2 images stereo	Left and right eye, each half the canvas width	body 1
4 people	2 × 2 grid of Center-eye views, one per person	bodies 1 to 4
2 people stereo	2 × 2 grid; each row is one person, left eye on the left, right eye on the right	body 1 (top row), body 2 (bottom row)
Projector 4K stereo (top quadrants)	Left eye top-left, right eye top-right, each a quarter of the canvas (Full HD at 3840 × 2160); the bottom half stays unused	body 1

In the wall’s editor the templates are buttons, for example Template: 2 people stereo; each replaces the current viewports. Canvas previews the arrangement. Use Add, Remove, Up and Down to edit the list of viewport children, and Edit slots to open a viewport’s component in the inspector. At most eight viewports are allowed. A wall without any viewport draws one full-canvas view from the SweetSpot.

Each eye sees the whole wall from its own tracked position; the canvas carries one area per eye.

Field	Default	Meaning
Label	Person 1 for the initial viewport	Name shown in the preview and tracking status
Rect (0–1)	x=0, y=0, w=1, h=1	Normalized canvas rectangle, with x/y measured from the upper-left; w/h must be positive, rectangles must fit inside the canvas and cannot overlap
Eye	Center	Left, Right or Center; Center gives one view without left/right eye separation
PosX, PosY, PosZ	Not connected	Signal slots for the tracked position in mm
QuatX, QuatY, QuatZ, QuatW	Not connected	Signal slots for the tracked orientation
Valid	Not connected	Signal slot that reports a valid pose
Frame	Not connected	Signal slot of the tracking heartbeat, for example `dtrack.frame`
EyeOffset	(0, -30, -80) mm	Offset from the tracked body to the eye centre, rotated with the tracked orientation
EyeSeparation	64 mm	0–200 mm; separation for Left and Right eyes
TimeoutMs	500 ms	50–10000 ms; maximum time without a changed heartbeat before fallback
Smoothing	0.3	0–0.95; higher values smooth head motion more, 0 applies it directly

Match the rectangles to the way the wall controller distributes the input image to eyes and people. The left and right viewports of one person connect to the same body and differ only in Eye.

Connect head tracking

Set up an ART DTrack or OptiTrack NatNet interface in CONNECT and connect the viewer to that gateway, so its tracking signals are available. Tracking is then an interface like any other: each value a viewport reads is a signal slot, connected to a signal the same way as any component slot — drag a signal onto the slot or pick it from the list. A slot that is not connected is simply empty; the viewer never guesses a signal from a name.

Tracking data reaches the wall through ordinary CONNECT signals and signal slots.

Fill from body

Connecting nine slots by hand is error-prone. Fill from body in a viewport’s row connects them in one step:

Choose the Pose group, for example dtrack.body1. The list offers every signal group that carries an orientation (.qw).
Choose the Heartbeat signal, for example dtrack.frame.
Click Fill from body.

The position, orientation and valid slots are connected to the group’s .x, .y, .z, .qx, .qy, .qz, .qw and .valid signals, the Frame slot to the heartbeat you chose. The whole fill is one undo step. A signal that does not exist is listed under Not found (slot left as is); that slot keeps its previous connection. The same fill is available to automation as the command wall.fillFromBody.

A stationary head stays fresh while the heartbeat changes. A missing heartbeat, an invalid pose, unconnected pose slots or a timed-out heartbeat make that viewport fall back to SweetSpot.

While the simulation is paused

During a simulation pause, live signal values do not reach the viewer. The wall therefore holds the last pose of each viewport and shows the state paused instead of falling back to the sweet spot after the timeout. After the simulation resumes, the heartbeat timeout applies again. Flysticks behave the same: their navigation stops, and no click, grab or navigation switch is triggered by button presses made during the pause.

Walls from earlier versions

Earlier versions of the wall configured viewports and Flysticks through signal prefixes. Such a wall still opens: its viewport geometry is kept and shown from the SweetSpot, its Flysticks are inactive, and the Problems panel reports Display wall tracking prefix ignored. Recreate the viewports with a template or Add, then connect their slots, for example with Fill from body.

Only the generated tracking signals count towards the CONNECT licence: one heartbeat plus the fields of the configured bodies and Flysticks. See Tracking signals and coordinates.

Preview, start and stop

Frame the model as required, then click Preview to check the areas on an ordinary monitor. The preview fits the configured canvas aspect ratio into the available space and labels the areas with their tracking state. It does not require fullscreen.

Click Start wall (fullscreen) for output at the configured canvas resolution. If the browser provides screen information, the viewer looks for a screen of that size, preferring a secondary display. Otherwise it uses fullscreen on the current display.

?wall=1 requests wall output and shows Start wall (fullscreen) and Preview; it waits for a user gesture.
?wall-preview requests a preview, which starts when a DisplayWall is available.
W starts or stops a wall when a DisplayWall exists and first-person navigation is inactive. F or Esc stops active wall output. Shortcuts are ignored in text fields and with Ctrl, Cmd or Alt.
Stop wall stops the output. Editing the configuration while output is active stops it; start again to apply the changes.

Use Flysticks

Add a WallFlystick (up to four) for each tracked controller under Flysticks in the wall’s editor and connect its slots, most easily with Fill from body: choose the Flystick’s pose group (for example dtrack.fly1), the heartbeat signal and, for the buttons, the signals for Select (press counter), Grab (held) and optionally Nav mode (press counter), plus Joystick X and Joystick Y. Flysticks work while the wall output or its preview is active. In CONNECT, FlystickCount must cover the controller you use: dtrack.fly3 needs a FlystickCount of at least 3.

Point — the Flystick casts a ray from its tracked pose. The object it points at is highlighted like a mouse hover; no ray is drawn. RayLength limits the ray. PointAxis sets the axis of the Flystick body that the ray follows.
Select — each new press on the SelectCount signal clicks the highlighted object, which selects it or triggers a clickable component. Every press counts once, even a very short one; holding does not repeat.
Grab — in Commissioning mode, hold the button connected to GrabHeld on a movable unit (MU) to move it along the ray, and release it to drop the MU. The same rules apply as for dragging with the mouse.
Fly — JoyX and JoyY move the view through the SpaceMouse navigation. JoystickDeadZone ignores small deflections. A press on NavCount switches between Orbit and Fly.

Field	Default	Meaning
PosX … QuatW, Valid, Frame	Not connected	The nine pose slots, as for a viewport; without them the Flystick is inactive
SelectCount	Not connected	Press counter of the select button (Int)
GrabHeld	Not connected	State of the grab button while held (Bool)
NavCount	Not connected	Press counter of the Orbit / Fly button (Int); optional
JoyX, JoyY	Not connected	Joystick axes (Float)
TimeoutMs	300 ms	50–5000 ms; maximum time without a changed heartbeat before the Flystick counts as lost
RayLength	50000 mm	1000–1000000 mm in tracking space; scaled with ModelScale like the wall
JoystickDeadZone	10 %	0–50 %
PointAxis	−Z	−Z, +Z, −Y, +Y, −X or +X of the Flystick’s own frame

A button signal connected to two of SelectCount, GrabHeld and NavCount is marked red on the affected slot rows (same signal as …) and in the wall’s Flystick list, and prevents the wall from starting. If the highlight does not follow the direction the Flystick points in, aim it at a known target on site and choose the PointAxis that hits it.

With several Flysticks, one points, selects and grabs at a time: the last one whose button was pressed. Every Flystick can fly. While one holds an MU, the others cannot take over.

A Flystick is lost when valid is off or its heartbeat is older than TimeoutMs. A held MU is then put back where the grab started, the highlight disappears and the Flystick’s navigation stops. When it returns, nothing is clicked or grabbed automatically; release a held grab button once before grabbing again. Stopping the wall output does the same for every Flystick before the mouse controls return.

Read the status

Each viewport’s Tracking line shows whether its slots are connected (Tracking slots, Frame slot empty or Sweet spot), its state, the heartbeat age and the connected heartbeat signal:

State	Meaning
fresh	A valid head pose and a recent, changing heartbeat are available
paused	The simulation is paused; the last pose is held
sweet spot	The configured fallback position is being used
too close	The eye is closer to the wall plane than Near, or behind it; its rendered position is clamped to Near

Output size mismatch (outputMismatch) reports the expected canvas dimensions and actual displayed dimensions. Output continues at the configured buffer size. Check the selected display and output sizing before using it on the wall.

Limits and known gaps

Wall output requires WebGL, a perspective camera and the standard viewer render backend. Stop XR, stereo output and first-person navigation before starting a wall.
Mouse camera controls are disabled during wall output. Pointing, selecting and grabbing with the mouse also stay disabled; use a Flystick instead. SpaceMouse navigation can move the view.
Picking by mouse within individual areas and a 3D cursor at the Flystick’s hit point are not available yet.
Post-processing such as ambient occlusion and bloom is bypassed. Screenshots and video captures remain mono.
NatNet provides head poses only; Flysticks require ART DTrack.
The axis conventions of DTrack and NatNet have so far been tested with simulated data only. Check them on site with a tracked target before the first presentation.

Operating a wall

A wall usually runs in a fixed installation, such as a powerwall or a projection room, on an output PC that stays on the plant network. For hosting the viewer there and the licence terms of a delivered installation, see Deployment Options, Licensing the Viewer and Appliance.