3D renderer

Unlike the 3D viewer, which displays the simulation’s 3D scene onto a window, the renderer exists to provide users the ability to render offscreen. This includes rendering RGB and depth images to either an array or to a file.

The renderer can be constructed by its builder (MjRenderer::builder) or directly with MjRenderer::new, however the latter offers less control. method. By default, RGB rendering is enabled, while depth rendering is disabled.

/* Build the renderer */.
let mut renderer = MjRenderer::builder()
    .width(0).height(0)  // set to width(0) and height(0) to set automatically based on <global offwidth="1920" offheight="1080"/>
    .num_visual_user_geom(5)  // maximum number of visual-only geoms as result of the user
    .num_visual_internal_geom(0)  // maximum number of visual-only geoms not as result of the user
    .font_scale(MjtFontScale::mjFONTSCALE_100)  // scale of the font drawn by OpenGL
    .rgb(true)  // rgb rendering
    .depth(true)  // depth rendering
    .camera(MjvCamera::default())  // default free camera
    .build(&model).expect("failed to initialize the renderer");

Attention

Renderer’s width and height must be equal to or less than the offscreen buffer size, configured during MuJoCo’s model definition:

<mujoco>
    <visual>
        <global offwidth="1920" offheight="1080"/>
    </visual>
    ...
</mujoco>

Much like the viewer, the renderer must also be synced with the simulation state, using MjRenderer::sync.

...
data.step();  // data is an instance of MjData.
renderer.sync(&mut data);
...

Note

MjRenderer::sync is similar to the MuJoco Python’s Renderer.update_scene method. Unlike the MuJoCo Python’s method, MuJoCo-rs’s implementation also performs rendering. This was done to make MjRenderer behave closer to the MjViewer.

After syncing, MjRenderer::rgb and MjRenderer::depth can be used to obtain a reference of the rendered image in correct shape, which needs to be specified via method’s const generic parameters, or MjRenderer::rgb_flat and MjRenderer::depth_flat can be used to obtain a flattened images.

To save rendered images to a file, MjRenderer::save_rgb and MjRenderer::save_depth can be used. These will encode the image into a uncompressed PNG format, where the RGB image will be 8 bits and the depth will be 16 bits. These are meant for visualization. To save depth data in float-32 bit format, which represents actual distance values from the camera, MjRenderer::save_depth_raw can be used.