Physical AI

Physical AI means AI systems that perceive and act on physical systems: a robot that balances on two legs, a gripper that adapts to the part in its jaws, a vehicle that finds its way around obstacles. Their controller is not a hand-written program. It is usually a neural network, called a policy, that was trained on examples. It reads sensor values and outputs commands for the actuators, many times per second.

Training and testing a policy on real hardware is slow, costly and risky: a robot that learns by falling over can break itself. So most of the work happens in simulation: training runs millions of simulated attempts, and tests run the finished policy through situations that would be dangerous or expensive on the real machine. Moving the policy from the simulator to the real machine is called sim-to-real. It only works if the simulator is physically close enough to the machine: the right masses, friction, joint limits and contact behaviour.

What realvirtual does with it today

realvirtual WEB runs trained policies inside a digital twin. It does not train them.

• A policy runs in the browser, next to the 3D model, the signals and the HMI of the same twin.
• Physics comes from detail physics (MuJoCo). The policy receives simulated sensor values and returns joint targets, and MuJoCo moves the robot with real forces and contacts.
• You command the robot through ordinary signals. The operator panel, the keyboard and a PLC all write the same signals, so a policy-driven machine can be connected to a control program like any other part of a model.
• The setup is data. The model names its robot, its policy files and its physics parameters, and the project carries those files, so nothing is hard-coded for one robot.

The runtime reads policies as ONNX files. The demo uses policies that were trained by third parties and are published under the Apache-2.0 licence.

Physical AI is a use case, not a level of its own. Today a learned controller runs on the detail physics level.

Where it fits among the simulation levels

On the Multi-Model Simulation page, the real-time approaches are levels that differ in detail and in the data they need. Physical AI is not a further level. It is a way of using the most detailed one.

• Detail physics (MuJoCo) is where policies run today. A legged robot or an adaptive gripper has many joints and contacts, and a policy only makes sense if those are simulated with forces. Kinematic motion cannot show whether a robot keeps its balance.
• The rest of the plant stays light. Conveyors, drives and PLC logic around the robot can remain kinematic, as described under Combining the approaches.
• Discrete-event simulation does not apply. It answers throughput questions, not questions about motion and control.

As on every physics level, choose the level your data supports. A policy tested against guessed masses and friction values only proves that it works in the simulator.

What you can try

• Demo: Walking robot (MuJoCo) - a two-legged robot walks through a field of obstacles it can push, trip over and fall against. A second policy stands it up again.

Outlook: realvirtual around the training loop

Planned, not available yet. realvirtual is meant to be the basis for Physical AI: running policies in the twin works today, and the training interface is planned as an open one. The engine parts are planned for the open AGPL core, and the bridge to Python as an open package, so you train with standard tools against the same twin. Training runs locally at your site, not in the browser: WebAssembly is too slow for the millions of simulation steps it needs. Camera images as policy input are planned for later. The sequence:

1. Build the scenario in the digital twin: machine, robot, sensors and PLC signals.
2. Export the model together with the task, the reward and the ranges for variation (mass, friction, part positions) for GPU training with standard tools such as MuJoCo MJX and MuJoCo Playground.
3. Import the trained policy back as an ONNX file. Running a compatible ONNX policy already works today; the demo does it.
4. Validate the policy in the real plant context: with the PLC, the sensors, disturbances and the cycle time.

The aim is a smaller sim-to-real gap, because the twin carries the real plant logic and not only the robot.

Status

Detail physics with MuJoCo is Beta. It belongs to the commercial tier and is included in the public demo project. The Editions page lists what each edition contains.

See also

• Multi-Model Simulation - all simulation levels and how they combine.
• Physics - rigid-body physics with Rapier.