Code samples

MuJoCo comes with several code samples providing useful functionality. Some of them are quite elaborate (simulate.cc in particular) but nevertheless we hope that they will help users learn how to program with the library.

testspeed

This code sample times the simulation of a given model. The timing is straightforward: the simulation of the passive dynamics (with optional control noise) is rolled-out for the specified number of steps, while collecting statistics about the number of contacts, scalar constraints, and CPU times from internal profiling. The results are then printed to the console. To simulate controlled dynamics instead of passive dynamics one can either install a control callback mjcb_control, or modify the code to set control signals explicitly, as explained in the simulation loop section below. This command-line utility is run with

testspeed [options] model

Where the command-line options and arguments are

Option	Default	Meaning
model	(required)	path to model (positional argument)
--nstep=N	10000	number of steps per rollout
--nthread=N	1	number of threads running parallel rollouts
--noisestd=X	0.01	scale of pseudo-random noise injected into actuators
--noiserate=X	0.1	rate of convergence to ctrl keyframe/midpoint
--nenginethread=N	0	number of worker threads in engine-internal threadpool
--solver=S	Newton	override constraint solver algorithm (PGS, CG, Newton)
--cone=C	Pyramidal	override friction cone type (Pyramidal, Elliptic)
--jacobian=J	Auto	override constraint Jacobian type (Dense, Sparse, Auto)
--integrator=I	Euler	override integration mode (Euler, RK4, Implicit, ImplicitFast)
--iterations=N	100	override solver iterations limit
--tolerance=X	1e-8	override solver convergence tolerance
--sleep_tolerance=X	1e-4	override sleep tolerance
--noslip_iterations=N	0	override noslip solver iterations limit

Notes:

• When nthread > 1 is specified, the code allocates a single mjModel and per-thread mjData, and runs nthread identical simulations in parallel. This tests performance with all cores active, as in Reinforcement Learning scenarios where samples are collected in parallel. The optimal nthread usually equals the number of logical cores.

• By default, the simulation starts from the model reference configuration with zero velocities. However, if a keyframe named “test” is present in the model, it is used as the initial state.

• The physics option override flags (such as --solver) only override the model settings if they are explicitly specified on the command line; otherwise, the model options configured in the XML file are preserved.

• The control noise arguments (noisestd and noiserate) prevent models from settling into a static state where, due to warmstarts, one can measure artificially faster simulation.

• When nenginethread > 0 is specified, an engine-internal thread pool is created with the specified number of worker threads (in addition to the calling thread), to speed up simulation of large scenes. Note that while it is possible to use both nthread and nenginethread, the scenarios for which one would want these different types of multithreading are usually mutually exclusive.

• For more repeatable performance statistics, run the tool with the performance governor on Linux, or the High Performance power plan on Windows, to reduce noise from CPU scaling.

• Many modern CPUs contain a mixture of “performance” and “efficiency” cores. Users should consider restricting the process to only run on the same type of cores for more interpretable performance statistics. This can be done via the taskset command on Linux, or the start /affinity command on Windows (processor affinity cannot be specified through documented API means on macOS).

simulate

This code sample is a fully-featured interactive simulator. It opens an OpenGL window using the platform-independent GLFW library, and renders the simulation state in it. There is built-in help, simulation statistics, profiler, sensor data plots. The model file can be specified as a command-line argument, or loaded at runtime using drag-and-drop functionality. This code sample uses the native UI to render various controls, and provides an illustration of how the new UI framework is intended to be used. Below is a screen-capture of simulate in action:

Interaction is done with the mouse; built-in help with a summary of available commands is available by pressing the F1 key. Briefly, an object is selected by left-double-click. The user can then apply forces and torques on the selected object by holding Ctrl and dragging the mouse. Dragging the mouse alone (without Ctrl) moves the camera. There are keyboard shortcuts for pausing the simulation, resetting, and re-loading the model file. The latter functionality is very useful while editing the model in an XML editor. The complete set of shortcuts is given in Shortcuts below.

Shortcuts

The F1 help overlay lists the most common commands. The tables below are the complete reference, and also apply to the Python viewer launched with mujoco.viewer, which is built on the same Simulate UI. Shortcuts that step or pause the simulation have no effect in passive mode, where stepping is driven by user code.

In addition to the shortcuts listed here, every UI control shows its own shortcut when the right mouse button is held over the UI panel.

Simulation and camera

Key	Action
Space	Play / pause
Right arrow	Step forward
Left arrow	Step backward, through the history buffer
+ (actually =)	Speed up
-	Slow down
Backspace	Reset
Ctrl C	Copy state to clipboard
Ctrl L	Reload the model
Ctrl A	Align the free camera
Esc	Switch to the free camera
[ / ]	Cycle down / up through the fixed cameras defined in the model
Page Up	Select the parent of the currently selected body
Tab / Shift Tab	Toggle the left / right UI panel

Panels and files

Key	Action
F1	Toggle the help overlay
F2	Toggle the info overlay
F3	Toggle the profiler
F4	Toggle the sensor plots
F5	Toggle fullscreen
F6	Cycle the frame visualization
F7	Cycle the label visualization
Ctrl M	Print the model to MJMODEL.TXT
Ctrl D	Print the data to MJDATA.TXT
Ctrl P	Save a screenshot
Ctrl Q	Quit
Alt + letter	Expand / collapse a UI section: F File, O Option, S Simulation, W Watch, P Physics, R Rendering, V Visualization, G Group enable, L Logging, J Joint, C Control, E Equality

Visibility groups

Key	Action
0 … 5	Toggle visibility of geom group 0 … 5
Shift 0 … Shift 5	Toggle visibility of site group 0 … 5

Visualization flags

These toggle the abstract visualization flags in the Rendering section, and correspond to the shortcuts declared in mjVISSTRING. Flags with no shortcut assigned (Select Point, the Flex flags and SDF iters) are omitted.

Key	Flag	Key	Flag
H	Convex Hull	X	Texture
J	Joint	Q	Camera
U	Actuator	,	Activation
Z	Light	V	Tendon
Y	Range Finder	E	Equality
I	Inertia	'	Scale Inertia
B	Perturb Force	O	Perturb Object
C	Contact Point	N	Island
F	Contact Force	P	Contact Split
T	Transparent	A	Auto Connect
M	Center of Mass	D	Static Body
;	Skin	`	Body Tree
\	Mesh Tree

Rendering flags

These toggle the OpenGL effects in the Rendering section, and correspond to the shortcuts declared in mjRNDSTRING. Flags with no shortcut assigned (Depth, Id Color and Cull Face) are omitted.

Key	Flag	Key	Flag
S	Shadow	W	Wireframe
R	Reflection	L	Additive
K	Skybox	G	Fog
/	Haze

Mouse

Action	Effect
Left drag	Orbit the camera
Right drag	Pan camera in vertical plane
Shift right drag	Pan camera in horizontal plane
Scroll, or middle drag	Zoom
Double-click	Select an object
Right double-click	Center the camera on the clicked point
Ctrl right double-click	Track the selected body
Ctrl drag	Rotate the selected object
Ctrl right drag	Translate object in vertical plane
Ctrl Shift right drag	Translate object in horizontal plane
Right-button hold over the UI	Show the shortcut of each UI control
Double-click a UI section title	Expand / collapse all sections

The code is long yet reasonably commented, so it is best to just read it. Here we provide a high-level overview. The main() function initializes both MuJoCo and GLFW, opens a window, and install GLFW callbacks for mouse and keyboard handling. Note that there is no render callback; GLFW puts the user in charge, instead of running a rendering loop behind the scenes. The main loop handles UI events and rendering. The simulation is handled in a background thread, which is synchronized with the main thread.

The mouse and keyboard callbacks perform whatever action is necessary. Many of these actions invoke functionality provided by MuJoCo’s abstract visualization mechanism. Indeed this mechanism is designed to be hooked to mouse and keyboard events more or less directly, and provides camera as well as perturbation control.

The profiler and sensor data plots illustrate the use of the mjr_figure function that can plot elaborate 2D figures with grids, annotation, axis scaling etc. The information presented in the profiler is extracted from the diagnostic fields of mjData. It is a very useful tool for tuning the parameters of the constraint solver algorithms. The outputs of the sensors defined in the model are visualized as a bar graph.

Note that the profiler shows timing information collected with high-resolution timers. On Windows, depending on the power settings, the OS may reduce the CPU frequency; this is because simulate.cc sleeps most of the time in order to slow down to realtime. This results in inaccurate timings. To avoid this problem, change the Windows power plan so that the minimum processor state is 100%.

compile

This code sample invokes the built-in parser and compiler. It implements all possible model conversions from (MJCF, URDF, MJB) format to (MJCF, MJB, TXT) format. Models are saved as MJCF as Model Encoding & Saving describes: by default with the values which compilation made of them, in a canonical subset of the format, so that MJCF-to-MJCF conversion will generally result in a different file; with savecompiled “false”, as they are written, which keeps their notation but not their comments, include elements and the elements which the parser expands. The TXT format is a human-readable road-map to the model. It cannot be loaded by MuJoCo, but can be a very useful aid during model development. It is in one-to-one correspondence with the compiled mjModel. Note also that one can use the function mj_printData to create a text file which is in one-to-one correspondence with mjData, although this is not done by the code sample.

If the input file is MJCF or URDF and the output file is empty, compilation is performed twice to measure the impact of the compiler’s asset cache. A detailed timing breakdown is printed for each compilation, showing total time, asset processing time (wall clock), and per-category CPU times for meshes and textures. These timings are read from the mjtCTimer fields via mjs_getTimer, which can be read programmatically after any call to mj_compile.

basic

This code sample is a minimal interactive simulator. The model file must be provided as command-line argument. It opens an OpenGL window using the platform-independent GLFW library, and renders the simulation state at 60 fps while advancing the simulation in real-time. Press Backspace to reset the simulation. The mouse can be used to control the camera: left drag to rotate, right drag to translate in the vertical plane, shift right drag to translate in the horizontal plane, scroll or middle drag to zoom.

The Visualization programming guide below explains how visualization works. This code sample is a minimal illustration of the concepts in that guide.

render

This code sample renders a model offscreen into either a single PNG image or a multi-frame stream of raw RGB24 frames that can be saved to a .rgb file or piped directly to tools such as ffmpeg. It supports both the modern Filament rendering backend (mjrf) with physically-based rendering (PBR) and the classic OpenGL renderer (mjrContext), running headless without creating an interactive window.

render model [output] [options]

Where the command-line arguments and options are

Option	Default	Meaning
model	(required)	path to model (positional argument)
output	(default)	output path (positional argument: .png, .rgb, or - for stdout; defaults to <model>.png, or <model>.rgb when --duration > 0)
--backend=B	filament	rendering backend (filament, classic)
--width=N	640	image width in pixels
--height=N	480	image height in pixels
--duration=T	0	recording duration in seconds (0 renders a single frame)
--fps=F	60	frames per second when --duration > 0
--camera=C	(free)	camera name or integer index (default: free camera)
--lookat=X,Y,Z	(center)	free camera lookat position
--distance=D	(fit)	free camera distance
--azimuth=A	90	free camera azimuth in degrees
--elevation=E	-45	free camera elevation in degrees
--key=K	(none)	keyframe name or integer index to load prior to rendering
--steps=N	0	number of simulation steps to advance before rendering
--geomgroup=G	111000	6-character string enabling/disabling geom groups 0-5 (e.g. 111111)
--sitegroup=S	111000	6-character string enabling/disabling site groups 0-5
--label=L	(none)	type of label (e.g. --label=geom, --label=body, --label=site)
--frame=F	(none)	type of coordinate frame (e.g. --frame=geom, --frame=body, --frame=site)
--<visflag>=0|1	(default)	toggle visualization flag declared in mjVISSTRING (e.g. --joint=1, --contactpoint=1)
--<rndflag>=0|1	(default)	toggle rendering flag declared in mjRNDSTRING (e.g. --shadow=0, --wireframe=1)

For example, a 5-second video at 60 frames per second can be piped directly into ffmpeg without writing raw frames to disk:

render humanoid.xml - --duration=5 --fps=60 --width=1280 --height=720 | \
  ffmpeg -f rawvideo -pixel_format rgb24 -video_size 1280x720 -framerate 60 \
         -i - -pix_fmt yuv420p video.mp4

Notes:

• Running render with no arguments (or with --help / -h) prints the help message.

• The non-optional model path and optional output path are positional arguments. If output is omitted, the output filename is derived from the model filename with .png for single-frame renders (--duration=0) or .rgb for multi-frame recordings (--duration > 0). Passing - as output streams raw RGB24 frames to standard output and sends status messages to standard error.

• Both --backend=filament and --backend=classic emit top-down RGB24 frames (row 0 at the top), so ffmpeg does not require a vflip filter.

• --backend=filament uses MuJoCo’s Filament C API to configure a scene from the model, setting up lighting, materials, and textures, and rendering via Filament’s offscreen pipeline. --backend=classic uses standard OpenGL offscreen rendering.

• The default camera is the model’s free camera looking at the center of the scene. Specifying --camera=name selects a named camera defined in the model XML, while integer indices 0, 1, ... select fixed cameras in order of definition. When using the free camera, its viewpoint can be adjusted via --lookat=x,y,z, --distance=d, --azimuth=a, and --elevation=e.

• If a keyframe is specified via --key, the model state is reset to that keyframe before evaluating kinematics and rendering. If no keyframe is specified and a keyframe named test exists in the model, it is used by default.

• --steps=N advances the simulation by N steps (after any keyframe reset) before rendering or starting a --duration recording, so a state or segment reached by simulating can be captured directly.

• Visualization and rendering flags can be individually toggled using normalized flag names from mjVISSTRING and mjRNDSTRING.

• Labels can be enabled using --label=L where L is a label type name from mjLABELSTRING (e.g. geom, body, joint) or an integer index.

• Coordinate frames can be rendered using --frame=F where F is a frame type name from mjFRAMESTRING (e.g. geom, body, site, world) or an integer index.