Architecture#
MuJoCo Studio’s architecture rests on three core ideas:
Decoupled simulation and visualization: Physics simulation and UI/rendering run on separate threads with no shared mutable state, communicating strictly through typed Snapshot and Event message channels. Any Python plugin therefore works unchanged against both a local NativeViewer window and a remote WebViewer browser session.
Composable extensibility via plugins: Built-in features—including the standard Studio UI (
studio_app.App), real-time physics stepping (step_control.StepControl), and trajectory history scrubbing (sim_history.SimHistory)—are implemented as modular viewer and simulation plugins that applications can mix, replace, or extend with custom UI panels, 3D overlays, and stepping backends.Shared C++ components: Core UI panels, 3D viewport interaction, physics step pacing, trajectory history buffers, and Filament rendering are implemented as modular C++ libraries that compile both to native machine code (with Python bindings) and to WebAssembly (with JavaScript bindings), ensuring consistent behavior across C++, Python, and browser targets.
Thread Model#
In Python, Studio separates physics simulation from rendering and UI presentation across two execution contexts
connected by message channels (SimEndpoint ↔ ViewerEndpoint), allowing the viewer and
simulation to run on separate threads or in independent processes:
This decoupled design provides three architectural guarantees:
Zero shared-state contention: Unlike the legacy passive viewer, which required user scripts to acquire
with viewer.lock():before mutating mjModel or mjData, Studio’s simulation thread exclusively owns the authoritative physics structs. After each step,ViewerHandle.sync()extracts the flat integration state vector via mj_getState (mjSTATE_INTEGRATION) into an immutableStateSnapshot(state, state_sig). The viewer thread applies the latest snapshot to its private copy of mjData via mj_setState and runs mj_forward locally at display refresh rate without blocking physics.Background viewer daemon thread: Both native and web Python launchers spawn the viewer loop on a background daemon thread (launch_thread.py) while running the simulation sync loop or caller
withblock on the calling thread. Unhandled exceptions on the viewer thread are logged immediately and re-raised on the simulation thread whenViewerHandle.close()runs.Unified plugin contract: Because viewer plugins and simulation plugins communicate exclusively through typed
SnapshotandEventmessages rather than shared memory, plugin classes work unchanged across native desktop windows and remote WebViewer browser sessions.
Python vs. C++#
Studio adapts its execution model to each runtime environment:
Python (
mujoco.studio): Simulation and viewer endpoints hold separate mjModel and mjData instances and communicate strictly through directionalSnapshotandEventchannels.C++ Desktop (mujoco::studio::App): Runs physics on a dedicated background thread synchronized with the main UI and rendering loop.
C++ WebAssembly (MuJoCo Live): Drives physics updates, Dear ImGui layout, and Filament WebGL rendering sequentially inside the browser’s single-threaded
requestAnimationFrameloop.
C++ Components#
Beneath both the Python SDK and the C++ applications sits a shared layer of reusable C++ components under studio/:
UI & interaction (ux/): Built-in Dear ImGui panels—model hierarchy, physics and rendering options, actuator controls, joint sliders, sensor and solver plots, the
SimHistorytimeline scrubber,Console, picture-in-picture views, and custom widgets inimgui_widgets.h—along with 3D mouse picking, camera motion, and body perturbation ininteraction.h. Exposed to Python viamujoco.studio.ux, which releases the Python GIL during C++ execution.Simulation state management (sim/): Real-time step pacing and control noise via
StepControl, ring-buffer trajectory recording and scrubbing viaSimHistory, profiling viaSimProfiler, and model lifecycle management viaModelHolder, wrapped by Python’sstep_controlandsim_historymodules.Rendering & windowing (hal/, render/): Hardware-abstracted Filament rendering via
FilamentRendererandModelScene, plus SDL2 and offscreen window management across OpenGL, Vulkan, and WebGL.
Because these C++ libraries compile to native binaries, bind to Python via pybind11, and compile to WebAssembly via
Emscripten with JavaScript bindings, any component or custom widget that needs to be shared across C++, Python, and
JavaScript/WebAssembly surfaces should be implemented in C++ and bound to Python and JS. Pure Python plugins execute on
the host CPU—streaming their UI draw lists to the browser in WebViewer—whereas C++
components can run natively on the host and client-side inside WebAssembly browser apps such as MuJoCo Live and the WebViewer browser client.
Message Passing#
Because the simulation and viewer loops run on separate threads with no shared mutable state (see Thread Model), all state updates and commands cross the endpoint boundary as typed messages defined in messages.py.
Snapshots vs. Events#
Communication is divided into two Message base classes with distinct delivery semantics. By convention, custom
message subclasses end in *Snapshot or *Event:
Base Class |
Channel Type |
Delivery Guarantee |
Examples |
|---|---|---|---|
|
|
Only the most recent snapshot of each type is delivered on drain. |
|
|
|
Every queued event is delivered in order; no event is dropped. |
|
Choosing a Type#
When designing custom messages for GUI widgets or plugin communication:
Use a
Snapshotfor continuous GUI options such as sliders, toggles, and settings where a message carries the complete current value and only the latest value matters, such asStepControlSnapshot,MjOptionSnapshot, or a plugin’s ownFooConfigSnapshot. If the viewer renders faster than the simulation steps while a user drags a slider, intermediate values coalesce automatically instead of queuing up.Use an
Eventfor discrete GUI button actions and one-shot commands such asResetEvent,SingleStepEvent,RequestPauseEvent,LoadModelEvent, andKeyframeEvent, and when distinct partial updates of the same class would clobber one another inSnapshotChannel’s per-type slot—for example,StateEvent(state, state_sig)applies targeted state overrides whosestate_sigbitmasks differ so each component is applied in order.
For chronological lifecycle event tables, diff-based synchronization, and handler priorities, see Python Plugins.