AI Assistant (Bridge)

The AI Bridge is the connection between the twin running in your browser and an AI assistant running on your machine. When enabled, it lets the assistant find parts, read drives and take screenshots. With your permission, the assistant can also jog axes, set signals, build layouts and connect PLC signals to the model.

The bridge itself runs locally. The viewer, realvirtual CONNECT and the bridge between them are on your machine. The model file is not uploaded. An assistant such as Claude Code sends what it reads from the twin (names, values, screenshots) to its AI service. The service processes this information like any other request you give it.

The CONNECT channel and AI control

Two things are separate:

• The CONNECT channel is the connection between this viewer tab and realvirtual CONNECT. When CONNECT serves the viewer (for example at http://localhost:5100/), the channel always opens. SolidCraft live sync, the server log and tab ownership work without any AI. On a page CONNECT does not serve (the hosted demo on web.realvirtual.io, your own website), the channel opens only when AI control is on or when you start a connection such as Connect to SolidCraft.
• AI control decides whether AI assistants and MCP clients may control this tab. While it is off, the tab offers them no viewer tools. It refuses every viewer tool call with an error that starts with AI_DISABLED:, including calls from assistants on this PC. CONNECT also enforces this restriction on its side. Switching AI control on or off takes effect at once, without a reload.

The channel can stay open while AI control is off. AI calls go to the active tab.

Switching it on

Open the panel from the AI control button in the left bar, or from Settings ▸ AI. On a phone or tablet the settings tab is the only route. Then turn on AI control (Allow AI agents / MCP clients to control this viewer).

Two things can happen on the way.

• No gateway answered. CONNECT hosts the connection, so the click offers the CONNECT download instead of an empty panel. For a viewer without CONNECT, see Without a gateway.
• First time on this device. A one-time dialog lists what an assistant can access: the whole scene, every PLC signal (including write access) and the simulation. It appears before AI control is switched on for the first time. Access from this machine needs no configuration. Access from another machine requires a valid API key. Got it and Configure… record your acknowledgement. Not now closes the dialog without agreement, and AI control stays off. The CONNECT channel section of the tab is visible before you answer.

The device stores your acknowledgement only after you give it explicitly. If the bridge’s access expands, it asks again.

Reading the panel

Settings ▸ AI has two parts. The CONNECT channel section at the top reports the connection itself:

Row	What it tells you
CONNECT channel	Connected, Connecting…, Not connected, or the cause when it fails (with a hint and Retry Now)
Why	Why the channel is open: served by CONNECT, opened for AI control, or opened for a connection (e.g. SolidCraft)
Target	Where the channel connects to, normally CONNECT on this machine

The AI control section below it covers the assistant:

Row	What it tells you
AI control	The switch, and On or Off
This tab	Active (or Active (agent tab)), or Standby — another tab is in control when several viewer tabs are open
AI client	Which assistant is attached
Last AI activity	How long ago the assistant last did something
Tools	How many tools this viewer offers: 0 (AI off) while AI control is off
Port	The port this browser is talking to
Bridge	The bridge process: id, port and how long it has been running. Useful when two are running by accident

With several viewer tabs open, one tab is in control; while any tab has AI control on, only such a tab can take control. CONNECT serves at most eight viewer tabs at once.

Below the status:

• Port is the port of the bridge. Leave it at 5100 for CONNECT. Change it only for the fallback described below.
• Pause / Resume stops and resumes accepting connections.
• Shutdown asks the bridge process to exit. It can only be restarted by the assistant, not from the browser.
• Registered Tools lists the tools currently available to the assistant. Server Log shows its activity.

Fast agent runs: turn off Show AI clicks in Settings ▸ AI, or use the finger icon beside New session in the AI prompt toolbar. This per-device setting is on by default. Off skips the button hover and press delays and the wait for Inspector rows; selection, Inspector opening, field reveal, Follow agent and result highlighting stay active. Agents can set it through web_execute with the command view.aiClicks and the args {on:false} ({on:true} restores the animations, omitting on toggles). The command returns {on}.

Seeing that it is working

The AI control button uses the accent colour while AI control is on and the tab is connected. Its tooltip reads AI control — off or AI control — on (N tools). During assistant activity, an indicator beside the button names the operation, such as AI · Drive list. It clears a few seconds after the last operation.

Connecting an assistant

CONNECT’s MCP endpoint is on by default, so there is usually nothing to enable on the gateway side. If it was switched off, turn it on in the CONNECT tray menu under MCP server ▸ Enabled and restart CONNECT. Register the endpoint with your assistant, restart the assistant, then switch AI control on in the panel, with the Port left at 5100.

Claude Code, and the Claude Code desktop application, speak to the endpoint directly:

{
  "mcpServers": {
    "realvirtual-CONNECT": {
      "type": "http",
      "url": "http://localhost:5100/mcp"
    }
  }
}

Claude Desktop (classic) can only launch local programs, so it needs a small relay in between. It requires Node.js on the machine:

{
  "mcpServers": {
    "realvirtual-CONNECT": {
      "command": "npx",
      "args": ["-y", "mcp-remote", "http://localhost:5100/mcp", "--allow-http"]
    }
  }
}

In Unity, Tools ▸ realvirtual ▸ Settings ▸ Configure Claude Desktop MCP writes both configurations for you.

Connections from a cloud assistant are not supported because they would require public access to the gateway.

Without a gateway

For a standalone viewer without CONNECT, use the fallback Node bridge. Build it once from the delivered sources and register it with the assistant in place of the endpoint above. Set the panel’s Port to 18714 for Claude Desktop or 18715 for Claude Code.

Use CONNECT where available. It shares the delivered HMI’s origin, needs no Node.js or build step, and includes the gateway’s tools.

Notes on a node

An assistant can record a finding on the relevant node, such as an axis’s role, actual limits or misleading part name.

Select an annotated node to read its note in the property inspector. The header shows the date, author and confidence level. Annotated nodes have a marker in the hierarchy.

• They are written by an assistant, not by you. The section is read-only, with no edit field or delete button.
• The confidence line matters. observed means measured or seen. inferred and unverified indicate uncertain findings. Notes record findings; they are not specifications.
• The date is the only staleness signal. Nothing checks a note against the model. An old note next to a changed machine stays where it is.
• The note travels inside the document. It is part of the GLB, so it follows the file. A re-export from Unity drops it.

AI diagnosis uses notes when you ask about an annotated node. Its answer draws on the manual, live state and findings from earlier sessions.

Quick Start

1. Start CONNECT on the same machine.
2. Register the endpoint with your assistant and restart it.
3. Open Settings ▸ AI in the viewer and check that CONNECT channel shows Connected.
4. Switch AI control on and read the one-time notice.
5. Check that AI client names your assistant and that Tools is not zero.
6. Start with a read-only request, such as “list the drives in this model”, and watch the activity indicator.

See also

• AI clients (MCP) — what an assistant can do, and the gateway switches that govern it
• Settings — the AI tab
• AI diagnosis — the other AI feature, which answers from your machine documentation
• Signal binding — the manual counterpart of assistant-driven binding